* [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call
@ 2018-09-13 12:20 Cedric Hombourger
2018-09-13 12:36 ` Henning Schild
0 siblings, 1 reply; 10+ messages in thread
From: Cedric Hombourger @ 2018-09-13 12:20 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
Make bitbake add proxy environment variables and preserve them when
calling debootstrap under sudo. This is required to get user-defined
proxies used while bootstraping Isar.
Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 4010307..a936e7f 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -171,7 +171,8 @@ isar_bootstrap() {
esac
shift
done
- sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
+ E="${@bb.utils.export_proxies(d)}"
+ sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
set -e
if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
if [ -e "${ROOTFSDIR}" ]; then
@@ -179,7 +180,6 @@ isar_bootstrap() {
umount -l "${ROOTFSDIR}/proc" || true
rm -rf "${ROOTFSDIR}"
fi
- E="${@bb.utils.export_proxies(d)}"
if [ ${IS_HOST} ]; then
${DEBOOTSTRAP} --verbose \
--variant=minbase \
--
2.11.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 12:20 [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call Cedric Hombourger
@ 2018-09-13 12:36 ` Henning Schild
2018-09-13 12:42 ` Hombourger, Cedric
0 siblings, 1 reply; 10+ messages in thread
From: Henning Schild @ 2018-09-13 12:36 UTC (permalink / raw)
To: Cedric Hombourger; +Cc: isar-users
Am Thu, 13 Sep 2018 14:20:58 +0200
schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
> Make bitbake add proxy environment variables and preserve them when
> calling debootstrap under sudo. This is required to get user-defined
> proxies used while bootstraping Isar.
>
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> 4010307..a936e7f 100644 ---
> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -171,7
> +171,8 @@ isar_bootstrap() { esac
> shift
> done
> - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> set -e
> if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> if [ -e "${ROOTFSDIR}" ]; then
> @@ -179,7 +180,6 @@ isar_bootstrap() {
> umount -l "${ROOTFSDIR}/proc" || true
> rm -rf "${ROOTFSDIR}"
> fi
> - E="${@bb.utils.export_proxies(d)}"
> if [ ${IS_HOST} ]; then
> ${DEBOOTSTRAP} --verbose \
> --variant=minbase \
So you are pulling that thing out of the sudo? How does that change
what is happening? One way or the other, the environment should be as
expected.
The other thing i was wondering, why is this export_proxies expression
in the flock-block twice?
If there is a difference in doing the export_proxies before "sudo
flock", please explain why it should be outside. And remove the second
inner export_proxies if possible.
Henning
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 12:36 ` Henning Schild
@ 2018-09-13 12:42 ` Hombourger, Cedric
2018-09-13 12:46 ` [PATCH v2] " Cedric Hombourger
0 siblings, 1 reply; 10+ messages in thread
From: Hombourger, Cedric @ 2018-09-13 12:42 UTC (permalink / raw)
To: Henning Schild; +Cc: isar-users
Hi Henning,
I was also curious about this export_proxies() thing and looked at its implementation as I was not understanding how the E variable was being used
export_proxies() expands to True if it succeeded. It is a python function that puts http_proxy and friends into the environment. When we execute
the shell block passed to sudo, the Python code will evidently have no effect. If unsure, take a look at the generated run.do_bootstrap script.
I missed the export_proxies() call we have towards the end of the sudo block. That's a good catch. Will update my patch
Cedric
-----Original Message-----
From: Henning Schild [mailto:henning.schild@siemens.com]
Sent: Thursday, September 13, 2018 2:36 PM
To: Hombourger, Cedric <Cedric_Hombourger@mentor.com>
Cc: isar-users@googlegroups.com
Subject: Re: [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call
Am Thu, 13 Sep 2018 14:20:58 +0200
schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
> Make bitbake add proxy environment variables and preserve them when
> calling debootstrap under sudo. This is required to get user-defined
> proxies used while bootstraping Isar.
>
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> 4010307..a936e7f 100644 ---
> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -171,7
> +171,8 @@ isar_bootstrap() { esac
> shift
> done
> - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> set -e
> if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> if [ -e "${ROOTFSDIR}" ]; then @@ -179,7 +180,6 @@
> isar_bootstrap() {
> umount -l "${ROOTFSDIR}/proc" || true
> rm -rf "${ROOTFSDIR}"
> fi
> - E="${@bb.utils.export_proxies(d)}"
> if [ ${IS_HOST} ]; then
> ${DEBOOTSTRAP} --verbose \
> --variant=minbase \
So you are pulling that thing out of the sudo? How does that change what is happening? One way or the other, the environment should be as expected.
The other thing i was wondering, why is this export_proxies expression in the flock-block twice?
If there is a difference in doing the export_proxies before "sudo flock", please explain why it should be outside. And remove the second inner export_proxies if possible.
Henning
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 12:42 ` Hombourger, Cedric
@ 2018-09-13 12:46 ` Cedric Hombourger
2018-09-13 13:09 ` Jan Kiszka
2018-09-25 17:39 ` Maxim Yu. Osipov
0 siblings, 2 replies; 10+ messages in thread
From: Cedric Hombourger @ 2018-09-13 12:46 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
Make bitbake add proxy environment variables and preserve them when
calling debootstrap under sudo. This is required to get user-defined
proxies used while bootstraping Isar.
Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 4010307..cfad136 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -171,7 +171,8 @@ isar_bootstrap() {
esac
shift
done
- sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
+ E="${@bb.utils.export_proxies(d)}"
+ sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
set -e
if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
if [ -e "${ROOTFSDIR}" ]; then
@@ -179,7 +180,6 @@ isar_bootstrap() {
umount -l "${ROOTFSDIR}/proc" || true
rm -rf "${ROOTFSDIR}"
fi
- E="${@bb.utils.export_proxies(d)}"
if [ ${IS_HOST} ]; then
${DEBOOTSTRAP} --verbose \
--variant=minbase \
@@ -233,7 +233,6 @@ isar_bootstrap() {
mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
mount -t proc none ${ROOTFSDIR}/proc
- E="${@bb.utils.export_proxies(d)}"
export DEBIAN_FRONTEND=noninteractive
chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
--
2.11.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 12:46 ` [PATCH v2] " Cedric Hombourger
@ 2018-09-13 13:09 ` Jan Kiszka
2018-09-13 13:15 ` Hombourger, Cedric
2018-09-25 17:39 ` Maxim Yu. Osipov
1 sibling, 1 reply; 10+ messages in thread
From: Jan Kiszka @ 2018-09-13 13:09 UTC (permalink / raw)
To: Cedric Hombourger, isar-users
Hi Cedric,
On 13.09.18 14:46, Cedric Hombourger wrote:
> Make bitbake add proxy environment variables and preserve them when
> calling debootstrap under sudo. This is required to get user-defined
> proxies used while bootstraping Isar.
Can you specify the error scenario a bit more precisely? We are building with
user-defined proxies frequently, and we do not need that change. While I have no
problem with saving one line of code, I would be good to understand the setup,
specifically as I requested to set up a proxies test scenario in the upstream
Q&A to avoid relying on the Siemens-internal Q&A here.
Thanks,
Jan
>
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index 4010307..cfad136 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -171,7 +171,8 @@ isar_bootstrap() {
> esac
> shift
> done
> - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> set -e
> if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> if [ -e "${ROOTFSDIR}" ]; then
> @@ -179,7 +180,6 @@ isar_bootstrap() {
> umount -l "${ROOTFSDIR}/proc" || true
> rm -rf "${ROOTFSDIR}"
> fi
> - E="${@bb.utils.export_proxies(d)}"
> if [ ${IS_HOST} ]; then
> ${DEBOOTSTRAP} --verbose \
> --variant=minbase \
> @@ -233,7 +233,6 @@ isar_bootstrap() {
> mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
> mount -t proc none ${ROOTFSDIR}/proc
>
> - E="${@bb.utils.export_proxies(d)}"
> export DEBIAN_FRONTEND=noninteractive
> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
>
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 13:09 ` Jan Kiszka
@ 2018-09-13 13:15 ` Hombourger, Cedric
2018-09-13 13:37 ` Henning Schild
0 siblings, 1 reply; 10+ messages in thread
From: Hombourger, Cedric @ 2018-09-13 13:15 UTC (permalink / raw)
To: Jan Kiszka; +Cc: isar-users
Hi Jan,
Are you sure your company mirror of debian repositories wasn't used in your builds?
I will repeat the tests that I did both with and without the changes.
I was traveling to Nurnberg earlier this week and unfortunately had a very slow and unreliable internet connection at our hotel
When checking squid access logs in realtime, I noticed that no requests were received while running do_bootstrap
I however agree that we should make sure we are fixing a real problem
I will therefore collect additional data
Cedric
-----Original Message-----
From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
Sent: Thursday, September 13, 2018 3:09 PM
To: Hombourger, Cedric <Cedric_Hombourger@mentor.com>; isar-users@googlegroups.com
Subject: Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
Hi Cedric,
On 13.09.18 14:46, Cedric Hombourger wrote:
> Make bitbake add proxy environment variables and preserve them when
> calling debootstrap under sudo. This is required to get user-defined
> proxies used while bootstraping Isar.
Can you specify the error scenario a bit more precisely? We are building with user-defined proxies frequently, and we do not need that change. While I have no problem with saving one line of code, I would be good to understand the setup, specifically as I requested to set up a proxies test scenario in the upstream Q&A to avoid relying on the Siemens-internal Q&A here.
Thanks,
Jan
>
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index 4010307..cfad136 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -171,7 +171,8 @@ isar_bootstrap() {
> esac
> shift
> done
> - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> set -e
> if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> if [ -e "${ROOTFSDIR}" ]; then @@ -179,7 +180,6 @@
> isar_bootstrap() {
> umount -l "${ROOTFSDIR}/proc" || true
> rm -rf "${ROOTFSDIR}"
> fi
> - E="${@bb.utils.export_proxies(d)}"
> if [ ${IS_HOST} ]; then
> ${DEBOOTSTRAP} --verbose \
> --variant=minbase \ @@ -233,7 +233,6
> @@ isar_bootstrap() {
> mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
> mount -t proc none ${ROOTFSDIR}/proc
>
> - E="${@bb.utils.export_proxies(d)}"
> export DEBIAN_FRONTEND=noninteractive
> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
>
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 13:15 ` Hombourger, Cedric
@ 2018-09-13 13:37 ` Henning Schild
2018-09-13 16:17 ` chombourger
0 siblings, 1 reply; 10+ messages in thread
From: Henning Schild @ 2018-09-13 13:37 UTC (permalink / raw)
To: Hombourger, Cedric; +Cc: Jan Kiszka, isar-users
Am Thu, 13 Sep 2018 13:15:03 +0000
schrieb "Hombourger, Cedric" <Cedric_Hombourger@mentor.com>:
> Hi Jan,
>
> Are you sure your company mirror of debian repositories wasn't used
> in your builds? I will repeat the tests that I did both with and
> without the changes.
Yes, i have never seen proxy issues in that step and never use our
internal mirror.
Henning
> I was traveling to Nurnberg earlier this week and unfortunately had a
> very slow and unreliable internet connection at our hotel When
> checking squid access logs in realtime, I noticed that no requests
> were received while running do_bootstrap
>
> I however agree that we should make sure we are fixing a real problem
> I will therefore collect additional data
>
> Cedric
>
> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> Sent: Thursday, September 13, 2018 3:09 PM
> To: Hombourger, Cedric <Cedric_Hombourger@mentor.com>;
> isar-users@googlegroups.com Subject: Re: [PATCH v2] isar-bootstrap:
> preserve environment in sudo'ed debootstrap call
>
> Hi Cedric,
>
> On 13.09.18 14:46, Cedric Hombourger wrote:
> > Make bitbake add proxy environment variables and preserve them when
> > calling debootstrap under sudo. This is required to get
> > user-defined proxies used while bootstraping Isar.
>
> Can you specify the error scenario a bit more precisely? We are
> building with user-defined proxies frequently, and we do not need
> that change. While I have no problem with saving one line of code, I
> would be good to understand the setup, specifically as I requested to
> set up a proxies test scenario in the upstream Q&A to avoid relying
> on the Siemens-internal Q&A here.
>
> Thanks,
> Jan
>
> >
> > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> > ---
> > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > index 4010307..cfad136 100644
> > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > @@ -171,7 +171,8 @@ isar_bootstrap() {
> > esac
> > shift
> > done
> > - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> > + E="${@bb.utils.export_proxies(d)}"
> > + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> > set -e
> > if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> > if [ -e "${ROOTFSDIR}" ]; then @@ -179,7 +180,6 @@
> > isar_bootstrap() {
> > umount -l "${ROOTFSDIR}/proc" || true
> > rm -rf "${ROOTFSDIR}"
> > fi
> > - E="${@bb.utils.export_proxies(d)}"
> > if [ ${IS_HOST} ]; then
> > ${DEBOOTSTRAP} --verbose \
> > --variant=minbase \ @@ -233,7
> > +233,6 @@ isar_bootstrap() {
> > mount -t devtmpfs -o mode=0755,nosuid devtmpfs
> > ${ROOTFSDIR}/dev mount -t proc none ${ROOTFSDIR}/proc
> >
> > - E="${@bb.utils.export_proxies(d)}"
> > export DEBIAN_FRONTEND=noninteractive
> > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y
> > \
>
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate
> Competence Center Embedded Linux
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 13:37 ` Henning Schild
@ 2018-09-13 16:17 ` chombourger
2018-09-25 10:05 ` chombourger
0 siblings, 1 reply; 10+ messages in thread
From: chombourger @ 2018-09-13 16:17 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 6666 bytes --]
Hi Henning,
I have reproduced the failure. Here's my setup:
* Host: Windows 10
* Guest: Debian 9 (running under VMWare)
* Isar: e231e88b447cdad1a233ad29ff23545bc50f398b (next)
Steps to reproduce (from the VM):
$ sudo route del -net 0.0.0.0 #
$ sudo route add -net 172.17.0.0 netmask 255.255.255.0 gw 192.168.20.2 $
ping -c 1 8.8.8.8
connect: network is unreachable # as expected, no direct connection to the
Internet $ export http_proxy=http://172.17.0.7:3128 $ export
https_proxy=http://172.17.0.7:3128
industrial@packer-debian-9-amd64:~/Projects/upstream/build-test$ bitbake
multiconfig:qemuamd64-stretch:isar-image-base
...
NOTE: Executing RunQueue Tasks
ERROR: mc:qemuamd64-stretch:isar-bootstrap-target-1.0-r0 do_bootstrap:
Function failed: do_bootstrap (log file is located at
/home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd64/isar-bootstrap-target/temp/log.do_bootstrap.2240)
ERROR: Logfile of failure stored in:
/home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd64/isar-bootstrap-target/temp/log.do_bootstrap.2240
Log data follows:
| DEBUG: Executing shell function do_bootstrap
| umount:
| /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
| 4/isar-bootstrap-target/rootfs/dev: mountpoint not found
| umount:
| /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
| 4/isar-bootstrap-target/rootfs/proc: mountpoint not found
| W: Target architecture is the same as host architecture; disabling
| QEMU support
| I: Running command: debootstrap --arch amd64 --verbose
| --variant=minbase --include=locales --components=main,contrib,non-free
| stretch
| /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
| 4/isar-bootstrap-target/rootfs http://ftp.de.debian.org/debian
| I: Retrieving InRelease
| I: Retrieving Release
| E: Failed getting release file
| http://ftp.de.debian.org/debian/dists/stretch/Release
| WARNING: exit code 1 from a shell command.
| ERROR: Function failed: do_bootstrap (log file is located at
| /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
| 4/isar-bootstrap-target/temp/log.do_bootstrap.2240)
ERROR: Task
(multiconfig:qemuamd64-stretch:/home/vmuser/Projects/upstream/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap)
failed with exit code '1'
NOTE: Tasks Summary: Attempted 12 tasks of which 10 didn't need to be rerun
and 1 failed.
Logs for squid do show that the proxy was used by bitbake to fetch sources
such as libhello They did not show any attempts to connect to Debian
repositories
I then added the previously submitted patch and the build went through (and
the squid logs did show a bunch of requests for debian.org)
Cedric
On Thursday, September 13, 2018 at 3:37:45 PM UTC+2, Henning Schild wrote:
>
> Am Thu, 13 Sep 2018 13:15:03 +0000
> schrieb "Hombourger, Cedric" <Cedric_H...@mentor.com <javascript:>>:
>
> > Hi Jan,
> >
> > Are you sure your company mirror of debian repositories wasn't used
> > in your builds? I will repeat the tests that I did both with and
> > without the changes.
>
> Yes, i have never seen proxy issues in that step and never use our
> internal mirror.
>
> Henning
>
> > I was traveling to Nurnberg earlier this week and unfortunately had a
> > very slow and unreliable internet connection at our hotel When
> > checking squid access logs in realtime, I noticed that no requests
> > were received while running do_bootstrap
> >
> > I however agree that we should make sure we are fixing a real problem
> > I will therefore collect additional data
> >
> > Cedric
> >
> > -----Original Message-----
> > From: Jan Kiszka [mailto:jan.k...@siemens.com <javascript:>]
> > Sent: Thursday, September 13, 2018 3:09 PM
> > To: Hombourger, Cedric <Cedric_H...@mentor.com <javascript:>>;
> > isar-...@googlegroups.com <javascript:> Subject: Re: [PATCH v2]
> isar-bootstrap:
> > preserve environment in sudo'ed debootstrap call
> >
> > Hi Cedric,
> >
> > On 13.09.18 14:46, Cedric Hombourger wrote:
> > > Make bitbake add proxy environment variables and preserve them when
> > > calling debootstrap under sudo. This is required to get
> > > user-defined proxies used while bootstraping Isar.
> >
> > Can you specify the error scenario a bit more precisely? We are
> > building with user-defined proxies frequently, and we do not need
> > that change. While I have no problem with saving one line of code, I
> > would be good to understand the setup, specifically as I requested to
> > set up a proxies test scenario in the upstream Q&A to avoid relying
> > on the Siemens-internal Q&A here.
> >
> > Thanks,
> > Jan
> >
> > >
> > > Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com <javascript:>>
>
> > > ---
> > > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
> > > 1 file changed, 2 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > index 4010307..cfad136 100644
> > > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > @@ -171,7 +171,8 @@ isar_bootstrap() {
> > > esac
> > > shift
> > > done
> > > - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> > > + E="${@bb.utils.export_proxies(d)}"
> > > + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> > > set -e
> > > if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> > > if [ -e "${ROOTFSDIR}" ]; then @@ -179,7 +180,6 @@
> > > isar_bootstrap() {
> > > umount -l "${ROOTFSDIR}/proc" || true
> > > rm -rf "${ROOTFSDIR}"
> > > fi
> > > - E="${@bb.utils.export_proxies(d)}"
> > > if [ ${IS_HOST} ]; then
> > > ${DEBOOTSTRAP} --verbose \
> > > --variant=minbase \ @@ -233,7
> > > +233,6 @@ isar_bootstrap() {
> > > mount -t devtmpfs -o mode=0755,nosuid devtmpfs
> > > ${ROOTFSDIR}/dev mount -t proc none ${ROOTFSDIR}/proc
> > >
> > > - E="${@bb.utils.export_proxies(d)}"
> > > export DEBIAN_FRONTEND=noninteractive
> > > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> > > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y
> > > \
> >
> > --
> > Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate
> > Competence Center Embedded Linux
> >
>
>
[-- Attachment #1.2: Type: text/html, Size: 10957 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 16:17 ` chombourger
@ 2018-09-25 10:05 ` chombourger
0 siblings, 0 replies; 10+ messages in thread
From: chombourger @ 2018-09-25 10:05 UTC (permalink / raw)
To: isar-users
[-- Attachment #1.1: Type: text/plain, Size: 6952 bytes --]
ping
On Thursday, September 13, 2018 at 6:17:33 PM UTC+2, chomb...@gmail.com
wrote:
>
> Hi Henning,
>
>
>
> I have reproduced the failure. Here's my setup:
>
>
>
> * Host: Windows 10
>
> * Guest: Debian 9 (running under VMWare)
>
> * Isar: e231e88b447cdad1a233ad29ff23545bc50f398b (next)
>
>
>
> Steps to reproduce (from the VM):
>
>
>
> $ sudo route del -net 0.0.0.0 #
>
> $ sudo route add -net 172.17.0.0 netmask 255.255.255.0 gw 192.168.20.2 $
> ping -c 1 8.8.8.8
>
> connect: network is unreachable # as expected, no direct connection to the
> Internet $ export http_proxy=http://172.17.0.7:3128 $ export https_proxy=
> http://172.17.0.7:3128
>
>
>
> industrial@packer-debian-9-amd64:~/Projects/upstream/build-test$ bitbake
> multiconfig:qemuamd64-stretch:isar-image-base
>
> ...
>
> NOTE: Executing RunQueue Tasks
>
> ERROR: mc:qemuamd64-stretch:isar-bootstrap-target-1.0-r0 do_bootstrap:
> Function failed: do_bootstrap (log file is located at
> /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd64/isar-bootstrap-target/temp/log.do_bootstrap.2240)
>
> ERROR: Logfile of failure stored in:
> /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd64/isar-bootstrap-target/temp/log.do_bootstrap.2240
>
> Log data follows:
>
> | DEBUG: Executing shell function do_bootstrap
>
> | umount:
>
> | /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
>
> | 4/isar-bootstrap-target/rootfs/dev: mountpoint not found
>
> | umount:
>
> | /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
>
> | 4/isar-bootstrap-target/rootfs/proc: mountpoint not found
>
> | W: Target architecture is the same as host architecture; disabling
>
> | QEMU support
>
> | I: Running command: debootstrap --arch amd64 --verbose
>
> | --variant=minbase --include=locales --components=main,contrib,non-free
>
> | stretch
>
> | /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
>
> | 4/isar-bootstrap-target/rootfs http://ftp.de.debian.org/debian
>
> | I: Retrieving InRelease
>
> | I: Retrieving Release
>
> | E: Failed getting release file
>
> | http://ftp.de.debian.org/debian/dists/stretch/Release
>
> | WARNING: exit code 1 from a shell command.
>
> | ERROR: Function failed: do_bootstrap (log file is located at
>
> | /home/vmuser/Projects/upstream/build-test/tmp/work/debian-stretch-amd6
>
> | 4/isar-bootstrap-target/temp/log.do_bootstrap.2240)
>
> ERROR: Task
> (multiconfig:qemuamd64-stretch:/home/vmuser/Projects/upstream/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb:do_bootstrap)
> failed with exit code '1'
>
> NOTE: Tasks Summary: Attempted 12 tasks of which 10 didn't need to be
> rerun and 1 failed.
>
>
>
> Logs for squid do show that the proxy was used by bitbake to fetch sources
> such as libhello They did not show any attempts to connect to Debian
> repositories
>
>
>
> I then added the previously submitted patch and the build went through
> (and the squid logs did show a bunch of requests for debian.org)
>
>
>
> Cedric
>
>
>
> On Thursday, September 13, 2018 at 3:37:45 PM UTC+2, Henning Schild wrote:
>>
>> Am Thu, 13 Sep 2018 13:15:03 +0000
>> schrieb "Hombourger, Cedric" <Cedric_H...@mentor.com>:
>>
>> > Hi Jan,
>> >
>> > Are you sure your company mirror of debian repositories wasn't used
>> > in your builds? I will repeat the tests that I did both with and
>> > without the changes.
>>
>> Yes, i have never seen proxy issues in that step and never use our
>> internal mirror.
>>
>> Henning
>>
>> > I was traveling to Nurnberg earlier this week and unfortunately had a
>> > very slow and unreliable internet connection at our hotel When
>> > checking squid access logs in realtime, I noticed that no requests
>> > were received while running do_bootstrap
>> >
>> > I however agree that we should make sure we are fixing a real problem
>> > I will therefore collect additional data
>> >
>> > Cedric
>> >
>> > -----Original Message-----
>> > From: Jan Kiszka [mailto:jan.k...@siemens.com]
>> > Sent: Thursday, September 13, 2018 3:09 PM
>> > To: Hombourger, Cedric <Cedric_H...@mentor.com>;
>> > isar-...@googlegroups.com Subject: Re: [PATCH v2] isar-bootstrap:
>> > preserve environment in sudo'ed debootstrap call
>> >
>> > Hi Cedric,
>> >
>> > On 13.09.18 14:46, Cedric Hombourger wrote:
>> > > Make bitbake add proxy environment variables and preserve them when
>> > > calling debootstrap under sudo. This is required to get
>> > > user-defined proxies used while bootstraping Isar.
>> >
>> > Can you specify the error scenario a bit more precisely? We are
>> > building with user-defined proxies frequently, and we do not need
>> > that change. While I have no problem with saving one line of code, I
>> > would be good to understand the setup, specifically as I requested to
>> > set up a proxies test scenario in the upstream Q&A to avoid relying
>> > on the Siemens-internal Q&A here.
>> >
>> > Thanks,
>> > Jan
>> >
>> > >
>> > > Signed-off-by: Cedric Hombourger <Cedric_H...@mentor.com>
>> > > ---
>> > > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
>> > > 1 file changed, 2 insertions(+), 3 deletions(-)
>> > >
>> > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
>> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
>> > > index 4010307..cfad136 100644
>> > > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
>> > > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
>> > > @@ -171,7 +171,8 @@ isar_bootstrap() {
>> > > esac
>> > > shift
>> > > done
>> > > - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
>> > > + E="${@bb.utils.export_proxies(d)}"
>> > > + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
>> > > set -e
>> > > if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
>> > > if [ -e "${ROOTFSDIR}" ]; then @@ -179,7 +180,6 @@
>> > > isar_bootstrap() {
>> > > umount -l "${ROOTFSDIR}/proc" || true
>> > > rm -rf "${ROOTFSDIR}"
>> > > fi
>> > > - E="${@bb.utils.export_proxies(d)}"
>> > > if [ ${IS_HOST} ]; then
>> > > ${DEBOOTSTRAP} --verbose \
>> > > --variant=minbase \ @@ -233,7
>> > > +233,6 @@ isar_bootstrap() {
>> > > mount -t devtmpfs -o mode=0755,nosuid devtmpfs
>> > > ${ROOTFSDIR}/dev mount -t proc none ${ROOTFSDIR}/proc
>> > >
>> > > - E="${@bb.utils.export_proxies(d)}"
>> > > export DEBIAN_FRONTEND=noninteractive
>> > > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
>> > > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y
>> > > \
>> >
>> > --
>> > Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate
>> > Competence Center Embedded Linux
>> >
>>
>>
[-- Attachment #1.2: Type: text/html, Size: 10884 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v2] isar-bootstrap: preserve environment in sudo'ed debootstrap call
2018-09-13 12:46 ` [PATCH v2] " Cedric Hombourger
2018-09-13 13:09 ` Jan Kiszka
@ 2018-09-25 17:39 ` Maxim Yu. Osipov
1 sibling, 0 replies; 10+ messages in thread
From: Maxim Yu. Osipov @ 2018-09-25 17:39 UTC (permalink / raw)
To: Cedric Hombourger, isar-users
On 9/13/18 3:46 PM, Cedric Hombourger wrote:
> Make bitbake add proxy environment variables and preserve them when
> calling debootstrap under sudo. This is required to get user-defined
> proxies used while bootstraping Isar.
Applied to the 'next',
Thanks,
Maxim.
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
> meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index 4010307..cfad136 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -171,7 +171,8 @@ isar_bootstrap() {
> esac
> shift
> done
> - sudo flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> + E="${@bb.utils.export_proxies(d)}"
> + sudo -E flock "${ISAR_BOOTSTRAP_LOCK}" -c "\
> set -e
> if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
> if [ -e "${ROOTFSDIR}" ]; then
> @@ -179,7 +180,6 @@ isar_bootstrap() {
> umount -l "${ROOTFSDIR}/proc" || true
> rm -rf "${ROOTFSDIR}"
> fi
> - E="${@bb.utils.export_proxies(d)}"
> if [ ${IS_HOST} ]; then
> ${DEBOOTSTRAP} --verbose \
> --variant=minbase \
> @@ -233,7 +233,6 @@ isar_bootstrap() {
> mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${ROOTFSDIR}/dev
> mount -t proc none ${ROOTFSDIR}/proc
>
> - E="${@bb.utils.export_proxies(d)}"
> export DEBIAN_FRONTEND=noninteractive
> chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade -y \
>
--
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2018-09-25 17:39 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-13 12:20 [PATCH] isar-bootstrap: preserve environment in sudo'ed debootstrap call Cedric Hombourger
2018-09-13 12:36 ` Henning Schild
2018-09-13 12:42 ` Hombourger, Cedric
2018-09-13 12:46 ` [PATCH v2] " Cedric Hombourger
2018-09-13 13:09 ` Jan Kiszka
2018-09-13 13:15 ` Hombourger, Cedric
2018-09-13 13:37 ` Henning Schild
2018-09-13 16:17 ` chombourger
2018-09-25 10:05 ` chombourger
2018-09-25 17:39 ` Maxim Yu. Osipov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox