HTTPs connection during bootstrap

HTTPs connection during bootstrap

[Ulrich Teichert]

[-- Attachment #1.1: Type: text/plain, Size: 7563 bytes --]

Hi,

after some teething problems, I've been able to build a bootable qemu ARM64 
image
with some of our packages for a proof of concept - thanks again to Anton.

Still open is getting a successful connection to an external apt-repository 
over HTTPs,
during bootstrapping which is secured by self signed certificates. 
Currently, I have to use
a reverse proxy (caddy - nice and simple setup) to circumvent the issue, 
and I would like to
get rid of it.

The error I'm getting at the moment when not using the reverse proxy is:

ERROR: mc:qemuarm64-trixie:isar-mmdebstrap-target-1.0-r0 do_bootstrap: 
ExecutionError('/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/temp/run.do_bootstrap.18929', 
25, None, None)
ERROR: Logfile of failure stored in: 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/temp/log.do_bootstrap.18929
Log data follows:
| DEBUG: Executing python function sstate_task_prefunc
| DEBUG: Python function sstate_task_prefunc finished
| DEBUG: Executing shell function do_bootstrap
| removed 
'/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/sources.list.d/bootstrap.list'
| 
'/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/apt-sources' 
-> 
'/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/sources.list.d/bootstrap.list'
| I: arm64 cannot be executed natively, but transparently using qemu-user 
binfmt emulation
| I: finding correct signed-by value...
| I: automatically chosen format: tar
| I: using 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch 
as tempdir
| W: Download is performed unsandboxed as root as file 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch/var/lib/apt/lists/partial 
couldn't be accessed by user _apt
| I: running --setup-hook in shell: sh -c 'mkdir -p 
"$1/var/cache/apt/archives/"' exec 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
| I: running --setup-hook in shell: sh -c 'flock -s 
/home/isar/isar-image/build/downloads/deb/debian-trixie.lock cp -n 
--no-preserve=owner \
|                       
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/dl_dir/var/cache/apt/archives/"*.deb 
\
|                       "$1/var/cache/apt/archives/" || true' exec 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
| I: running special hook: upload 
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/apt-preferences" 
/etc/apt/preferences.d/bootstrap
| I: running special hook: upload 
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/apt-sources-init" 
/etc/apt/sources-list
| I: running special hook: upload 
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/locale" 
/etc/locale
| I: running --setup-hook in shell: sh -c 'mkdir -p 
"$1/etc/apt/trusted.gpg.d"' exec 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
| I: running special hook: sync-in 
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/trusted.gpg.d" 
/etc/apt/trusted.gpg.d
| I: running --setup-hook in shell: sh -c 'install -v -m755 
"/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/chroot-setup.sh" 
"$1/chroot-setup.sh"' exec 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
| 
'/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/chroot-setup.sh' 
-> 
'/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch/chroot-setup.sh'
| I: running apt-get update...
| Ign:1 https://XXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
| Get:2 http://deb.debian.org/debian trixie InRelease [140 kB]
| Get:3 http://deb.debian.org/debian-security trixie-security InRelease 
[43.4 kB]
| Get:4 http://deb.debian.org/debian trixie-updates InRelease [47.3 kB]
| Get:5 http://deb.debian.org/debian trixie/non-free Sources [75.9 kB]
| Get:6 http://deb.debian.org/debian trixie/contrib Sources [52.3 kB]
| Get:7 http://deb.debian.org/debian trixie/main Sources [10.5 MB]
| Get:8 http://deb.debian.org/debian trixie/non-free-firmware Sources [6552 
B]
| Get:9 http://deb.debian.org/debian trixie/non-free-firmware arm64 
Packages [6484 B]
| Get:10 http://deb.debian.org/debian trixie/contrib arm64 Packages [48.4 
kB]
| Get:11 http://deb.debian.org/debian trixie/non-free arm64 Packages [74.4 
kB]
| Get:12 http://deb.debian.org/debian trixie/main arm64 Packages [9607 kB]
| Ign:1 https://XXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
| Get:13 http://deb.debian.org/debian-security 
trixie-security/non-free-firmware Sources [696 B]
| Get:14 http://deb.debian.org/debian-security trixie-security/main Sources 
[132 kB]
| Get:15 http://deb.debian.org/debian-security trixie-security/main arm64 
Packages [127 kB]
| Get:16 http://deb.debian.org/debian trixie-updates/main Sources [2788 B]
| Get:17 http://deb.debian.org/debian trixie-updates/main arm64 Packages 
[5404 B]
| Ign:1 https://XXXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
| Err:1 https://XXXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
|   SSL connection failed: error:0A000086:SSL routines::certificate verify 
failed / Success [IP: A.B.C.D 443]
| Fetched 20.9 MB in 7s (2899 kB/s)
| Reading package lists...
| E: Failed to fetch 
https://XXXXX.kumkeo.local/trixie/latest/dists/trixie/InRelease  SSL 
connection failed: error:0A000086:SSL routines::certificate verify failed / 
Success [IP: A.B.C.D 443]
| E: Some index files failed to download. They have been ignored, or old 
ones used instead.
| E: apt-get update --error-on=any -oAPT::Status-Fd=<$fd> 
-oDpkg::Use-Pty=false failed: process exited with 100 and error in console 
output
| W: hooklistener errored out: E: received eof on socket
| 
| I: main() received signal PIPE: waiting for setup...
| I: removing tempdir 
/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch...
| E: mmdebstrap failed to run
ERROR: Task 
(mc:qemuarm64-trixie:/home/isar/isar-image/isar/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb:do_bootstrap) 
failed with exit code '1'
NOTE: Tasks Summary: Attempted 136 tasks of which 135 didn't need to be 
rerun and 1 failed.
 
Summary: 1 task failed:
  
mc:qemuarm64-trixie:/home/isar/isar-image/isar/meta/recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb:do_bootstrap
Summary: There was 1 ERROR message, returning a non-zero exit code.

(internal hostname replaced by XXXXX, IP by A.B.C.D)

What would be the best way to inject the missing certificates into the 
bootstrapping
process?

Thanks in advance for every suggestion,
Uli

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/3a6bc2de-5694-4a72-90fd-6fcb5a62587en%40googlegroups.com.

[-- Attachment #1.2: Type: text/html, Size: 8372 bytes --]

Re: HTTPs connection during bootstrap

['Jan Kiszka' via isar-users]

On 28.04.26 08:58, Ulrich Teichert wrote:
> Hi,
> 
> after some teething problems, I've been able to build a bootable qemu
> ARM64 image
> with some of our packages for a proof of concept - thanks again to Anton.
> 
> Still open is getting a successful connection to an external apt-
> repository over HTTPs,
> during bootstrapping which is secured by self signed certificates.
> Currently, I have to use
> a reverse proxy (caddy - nice and simple setup) to circumvent the issue,
> and I would like to
> get rid of it.
> 
> The error I'm getting at the moment when not using the reverse proxy is:
> 
> ERROR: mc:qemuarm64-trixie:isar-mmdebstrap-target-1.0-r0 do_bootstrap:
> ExecutionError('/home/isar/isar-image/build/tmp/work/debian-trixie-
> arm64/isar-mmdebstrap-target/1.0-r0/temp/run.do_bootstrap.18929', 25,
> None, None)
> ERROR: Logfile of failure stored in: /home/isar/isar-image/build/tmp/
> work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/temp/
> log.do_bootstrap.18929
> Log data follows:
> | DEBUG: Executing python function sstate_task_prefunc
> | DEBUG: Python function sstate_task_prefunc finished
> | DEBUG: Executing shell function do_bootstrap
> | removed '/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/
> isar-mmdebstrap-target/1.0-r0/sources.list.d/bootstrap.list'
> | '/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-
> mmdebstrap-target/1.0-r0/apt-sources' -> '/home/isar/isar-image/build/
> tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/
> sources.list.d/bootstrap.list'
> | I: arm64 cannot be executed natively, but transparently using qemu-
> user binfmt emulation
> | I: finding correct signed-by value...
> | I: automatically chosen format: tar
> | I: using /home/isar/isar-image/build/tmp/work/debian-trixie-arm64/
> isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch as tempdir
> | W: Download is performed unsandboxed as root as file /home/isar/isar-
> image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/
> tempdir/mmdebstrap.3tADUZToch/var/lib/apt/lists/partial couldn't be
> accessed by user _apt
> | I: running --setup-hook in shell: sh -c 'mkdir -p "$1/var/cache/apt/
> archives/"' exec /home/isar/isar-image/build/tmp/work/debian-trixie-
> arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
> | I: running --setup-hook in shell: sh -c 'flock -s /home/isar/isar-
> image/build/downloads/deb/debian-trixie.lock cp -n --no-preserve=owner \
> |                       "/home/isar/isar-image/build/tmp/work/debian-
> trixie-arm64/isar-mmdebstrap-target/1.0-r0/dl_dir/var/cache/apt/
> archives/"*.deb \
> |                       "$1/var/cache/apt/archives/" || true' exec /
> home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-
> target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
> | I: running special hook: upload "/home/isar/isar-image/build/tmp/work/
> debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/apt-preferences" /etc/
> apt/preferences.d/bootstrap
> | I: running special hook: upload "/home/isar/isar-image/build/tmp/work/
> debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/apt-sources-init" /
> etc/apt/sources-list
> | I: running special hook: upload "/home/isar/isar-image/build/tmp/work/
> debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/locale" /etc/locale
> | I: running --setup-hook in shell: sh -c 'mkdir -p "$1/etc/apt/
> trusted.gpg.d"' exec /home/isar/isar-image/build/tmp/work/debian-trixie-
> arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
> | I: running special hook: sync-in "/home/isar/isar-image/build/tmp/
> work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/trusted.gpg.d" /
> etc/apt/trusted.gpg.d
> | I: running --setup-hook in shell: sh -c 'install -v -m755 "/home/isar/
> isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-
> target/1.0-r0/chroot-setup.sh" "$1/chroot-setup.sh"' exec /home/isar/
> isar-image/build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-
> target/1.0-r0/tempdir/mmdebstrap.3tADUZToch
> | '/home/isar/isar-image/build/tmp/work/debian-trixie-arm64/isar-
> mmdebstrap-target/1.0-r0/chroot-setup.sh' -> '/home/isar/isar-image/
> build/tmp/work/debian-trixie-arm64/isar-mmdebstrap-target/1.0-r0/
> tempdir/mmdebstrap.3tADUZToch/chroot-setup.sh'
> | I: running apt-get update...
> | Ign:1 https://XXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
> | Get:2 http://deb.debian.org/debian trixie InRelease [140 kB]
> | Get:3 http://deb.debian.org/debian-security trixie-security InRelease
> [43.4 kB]
> | Get:4 http://deb.debian.org/debian trixie-updates InRelease [47.3 kB]
> | Get:5 http://deb.debian.org/debian trixie/non-free Sources [75.9 kB]
> | Get:6 http://deb.debian.org/debian trixie/contrib Sources [52.3 kB]
> | Get:7 http://deb.debian.org/debian trixie/main Sources [10.5 MB]
> | Get:8 http://deb.debian.org/debian trixie/non-free-firmware Sources
> [6552 B]
> | Get:9 http://deb.debian.org/debian trixie/non-free-firmware arm64
> Packages [6484 B]
> | Get:10 http://deb.debian.org/debian trixie/contrib arm64 Packages
> [48.4 kB]
> | Get:11 http://deb.debian.org/debian trixie/non-free arm64 Packages
> [74.4 kB]
> | Get:12 http://deb.debian.org/debian trixie/main arm64 Packages [9607 kB]
> | Ign:1 https://XXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
> | Get:13 http://deb.debian.org/debian-security trixie-security/non-free-
> firmware Sources [696 B]
> | Get:14 http://deb.debian.org/debian-security trixie-security/main
> Sources [132 kB]
> | Get:15 http://deb.debian.org/debian-security trixie-security/main
> arm64 Packages [127 kB]
> | Get:16 http://deb.debian.org/debian trixie-updates/main Sources [2788 B]
> | Get:17 http://deb.debian.org/debian trixie-updates/main arm64 Packages
> [5404 B]
> | Ign:1 https://XXXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
> | Err:1 https://XXXXXXXXX.kumkeo.local/trixie/latest trixie InRelease
> |   SSL connection failed: error:0A000086:SSL routines::certificate
> verify failed / Success [IP: A.B.C.D 443]
> | Fetched 20.9 MB in 7s (2899 kB/s)
> | Reading package lists...
> | E: Failed to fetch https://XXXXX.kumkeo.local/trixie/latest/dists/
> trixie/InRelease  SSL connection failed: error:0A000086:SSL
> routines::certificate verify failed / Success [IP: A.B.C.D 443]
> | E: Some index files failed to download. They have been ignored, or old
> ones used instead.
> | E: apt-get update --error-on=any -oAPT::Status-Fd=<$fd> -oDpkg::Use-
> Pty=false failed: process exited with 100 and error in console output
> | W: hooklistener errored out: E: received eof on socket
> |
> | I: main() received signal PIPE: waiting for setup...
> | I: removing tempdir /home/isar/isar-image/build/tmp/work/debian-
> trixie-arm64/isar-mmdebstrap-target/1.0-r0/tempdir/mmdebstrap.3tADUZToch...
> | E: mmdebstrap failed to run
> ERROR: Task (mc:qemuarm64-trixie:/home/isar/isar-image/isar/meta/
> recipes-core/isar-mmdebstrap/isar-mmdebstrap-target.bb:do_bootstrap)
> failed with exit code '1'
> NOTE: Tasks Summary: Attempted 136 tasks of which 135 didn't need to be
> rerun and 1 failed.
>  
> Summary: 1 task failed:
>   mc:qemuarm64-trixie:/home/isar/isar-image/isar/meta/recipes-core/isar-
> mmdebstrap/isar-mmdebstrap-target.bb:do_bootstrap
> Summary: There was 1 ERROR message, returning a non-zero exit code.
> 
> (internal hostname replaced by XXXXX, IP by A.B.C.D)
> 
> What would be the best way to inject the missing certificates into the
> bootstrapping
> process?

Bootstrapping is done within the environment of your host or kas-isar in
case you use the build container. So, one way is to enrich the
appropriate environment with that special certificate prior to starting
the build.

Another one is to explore the extension of do_apt_config_prepare of the
bootstrap class with setting for
https://manpages.debian.org/trixie/apt/apt-transport-https.1.en.html.

There is no convenient way of configuring this via Isar variables
because that case is too uncommon. Normally, one signs the repo itself,
and can thus disable/ignore transport security.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/2b9c7c31-2ff0-41c3-8e66-8f9a86199252%40siemens.com.

AW: Re: HTTPs connection during bootstrap

[Ulrich Teichert]

Hi Jan,

[del]
>> What would be the best way to inject the missing certificates into the
>> bootstrapping
>> process?

>Bootstrapping is done within the environment of your host or kas-isar in
>case you use the build container. So, one way is to enrich the
>appropriate environment with that special certificate prior to starting
>the build.

Right, simple and works perfectly. Good to know that the host environment
is simply passed through.

>Another one is to explore the extension of do_apt_config_prepare of the
>bootstrap class with setting for
>https://manpages.debian.org/trixie/apt/apt-transport-https.1.en.html.

OK, for the moment I'm fine with modifying the host environment, but
I may come back to that later if we have to deal with more than our
own repository.

>There is no convenient way of configuring this via Isar variables
>because that case is too uncommon. Normally, one signs the repo itself,
>and can thus disable/ignore transport security.

I can't rule out that we will have to deal with repositories ouside of our
organisation in the future, so using one security layer more may become
necessary, but agreed: currently this is just overkill (but our IT department loves it...),

thanks,
Uli

Mit freundlichen Grüßen / Best regards


Dipl.-Inform. Ulrich Teichert
Senior Software Engineer



Phone +49 431 375938-0
_____________________________________

e.bs kumkeo GmbH
Am Kiel-Kanal 1
24106 Kiel, Deutschland

kumkeo.de

Rechnungen bitte an e.bs kumkeo GmbH, Heidenkampsweg 82a, 20097 Hamburg

Geschäftsführer Michael Leitner, Günter Hagspiel
Registergericht Amtsgericht Hamburg
Registernummer HRB 187712
USt-Idnr. DE449906070


-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/46f2271795ae485db5d2a840699875c6%40kumkeo.de.