Re: [PATCH v2 0/5] Debootstrap integration

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-23 08:40, Claudius Heine wrote:
> Hi Alex,
> 
> On 02/22/2018 03:21 PM, Alexander Smirnov wrote:
>> On 02/22/2018 04:18 PM, claudius.heine.ext@siemens.com wrote:
>>> From: Claudius Heine <ch@denx.de>
>>>
>>> Hi,
>>>
>>> this is the new rebased and fixed version of this patchset.
>>>
>>> Cheers,
>>> Claudius
>>>
>>> Changes from v1:
>>>    - rebased to current next
>>>    - added unmounting of /dev and /proc at the end of do_rootfs in
>>> isar-image-base
>>>      This was necessary for image file creation, otherwise it tried to
>>>      package stuff from the host.
>>>    - added 'isar-bootstrap:do_deploy' dependency to do_rootfs task in
>>>      image.bbclass.
>>>    - Changed 'RFSDIR' variable name to 'ROOTFSDIR' in
>>>      isar-bootstrap-helper.bbclass to be consistent with the variable
>>>      name in isar-bootstrap.bb
>>>    - Moved 'isar-apt' apt-preference settings from isar-bootstrap.bb to
>>>      isar-bootstrap-helper.bbclass
>>>    - Removed '--no-install-recommends' parameters in favor of adding the
>>>      apt configuration in 'isar-bootstrap.bb'
>>>    - unmount and remove possible existing bootstraped directories before
>>>      bootstrapping in do_bootstrap task of 'isar-bootstrap.bb'
>>>    - changed from /dev ro bind mounts to mounting devtmpfs to /dev to be
>>>      consistent with the other places where /dev is mounted.
>>>
>>
>> $ time bitbake multiconfig:qemuarm-wheezy:isar-image-base
>> multiconfig:qemuarm-jessie:isar-image-base
>> multiconfig:qemuarm-stretch:isar-image-base
>> multiconfig:qemui386-jessie:isar-image-base
>> multiconfig:qemui386-stretch:isar-image-base
>> multiconfig:qemuamd64-jessie:isar-image-base
>> multiconfig:qemuamd64-stretch:isar-image-base
>> multiconfig:rpi-jessie:isar-image-base
>> multiconfig:qemuarm64-stretch:isar-image-base
>>
>> ERROR: mc:rpi-jessie:isar-bootstrap-1.0-r0 do_bootstrap: Function
>> failed: do_bootstrap (log file is located at
>> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_bootstrap.29056)
>>
>> ERROR: Logfile of failure stored in:
>> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_bootstrap.29056
>>
>> ERROR: Task
>> (multiconfig:rpi-jessie:/home/builder/isar/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb:do_bootstrap)
>> failed with exit code '1'
>>
>> builder@zbook:~/isar$ cat
>> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_bootstrap.29056
>>
>> DEBUG: Executing shell function do_bootstrap
>> I: Running command: debootstrap --arch armhf --foreign --verbose
>> --variant minbase --components=main,contrib,non-free,firmware jessie
>> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap/rootfs
>> http://archive.raspbian.org/raspbian
>> I: Retrieving Release
>> I: Retrieving Release.gpg
>> I: Checking Release signature
>> E: Release signed by unknown key (key id 9165938D90FDDD2E)
>> WARNING: exit code 1 from a shell command.
>> ERROR: Function failed: do_bootstrap (log file is located at
>> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/isar-bootstrap/temp/log.do_bootstrap.29056)
> 
> Thx.
> 
> Is that the only issue that occured? In other words do all the other
> targets where the key was available on the host run through without any
> problems and the resulting image looks good? I am just asking to
> decrease iteration over the testing-fixing cycle. If that is the only
> problem, then I guess that this patchset can be merged after I
> successfully fixed the usage of repository keys, right?

There are some more issues, will comment on them in the patches.

This one here consists of several problems. First, we are missing the
Raspbian repo key. The question is how to obtain it. We could do this:

---8<---

From: Jan Kiszka <jan.kiszka@siemens.com>
Subject: [PATCH] Carry Raspbian public key ring

debootstrap only contains the Debian keys and requires a specific one
for other bootstrap repositories. One of them we support is Raspbian.

This imports the Raspbian public key in GPG format and makes them
available for bootstrapping by defining DISTRO_KEYRING in the distro
conf.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta-isar/conf/distro/raspbian-jessie.conf               |   1 +
 .../isar-bootstrap/files/raspbian.public.key.gpg         | Bin 0 -> 1225 bytes
 meta/recipes-core/isar-bootstrap/isar-bootstrap.bb       |  10 ++++++++--
 3 files changed, 9 insertions(+), 2 deletions(-)
 create mode 100644 meta/recipes-core/isar-bootstrap/files/raspbian.public.key.gpg

diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf
index 51d3023..89b769f 100644
--- a/meta-isar/conf/distro/raspbian-jessie.conf
+++ b/meta-isar/conf/distro/raspbian-jessie.conf
@@ -9,3 +9,4 @@ DISTRO_COMPONENTS ?= "main contrib non-free firmware"
 DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian"
 DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh"
 DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv"
+DISTRO_KEYRING ?= "raspbian.public.key.gpg"
diff --git a/meta/recipes-core/isar-bootstrap/files/raspbian.public.key.gpg b/meta/recipes-core/isar-bootstrap/files/raspbian.public.key.gpg
new file mode 100644
index 0000000000000000000000000000000000000000..c438c3e4c06d6c679db3071c9bdda71416a99b35
GIT binary patch
literal 1225
zcmV;)1UCDb0SyFCc*1D`2msGpm>Th%h_P)c%5B{Gj^+H_Vx<@C+wVtcMV`p^QtJm(
zbt-pSjSf2f>lpyXa#ruMvl*a~bH)1r4uc;)MCe)%sbh>c#z@(>gCXt%zmPi`IKcBc
z1hyn0(A*d@ut#D`JhgPk_BZ_ED2q-|uPxdFH$h_Xd)JqP<uBn?EkN&+M?UDNvoCH9
z_6t^_4fwd%ubA#?DHSo{@}N$IlSYBsw&O)>=pRi8Y_L0PW&6Wj^RQBVSOQG@Z)vw5
z)exaU-m)Kpxmhg;7T+P}`5c0hS8Ka5QzA$ID<@INY+w(z2S*Q3r5UcakumY^UJ^D`
z+EKZdQlTX$@Azb2fMoy?0RRECN=<2NWgt{&Z*6dMZ*CwcQeks&Vr6o2c_2_}AVg(i
zX<=?4VRCI~W*|XQO?EabRz^@MAUthwbZBpFaC2{NKxb`XX>2ZIZ*4w_0XPH`0RjLb
z1p-fa!f65<0|pBT2nPcK1{DYb2?`4Y76JnS0v-VZ7k~f?2@sKGlZ}x5-7cXD2mr+U
z<@g5B*u))XINMIJq>orm-9Iyr{EI)wPl244p;iBDNtkx%)qpnz41HSJ^of-DA?Fs$
zm>QB{Z>ghd<eXmD&EOi{rqH)&@L`uA6tdm7ukf(qzB5>xn0m6!0>Y-u?Z%Ez7>lWg
z^F=L#1%)Zp9CcLQmnBHe5P#VlVw8Jhr!m<WH<pX61;sI^cWf;jpP~41lF2F`C=ATk
zW^JA%$7bZxia4Sp-7SX*vUCv!;^G{*#1Y6BuHL?I3QjNhy&C*>c!sg|olpAY>`-+i
zrwJ(19;YDJ)c^R?e1udv)D)L=(q0(^TRxzF|Ez%bG)#<Bp)M5YcUuT~VHLRn4Fpek
z!f62r0H6$Ppws;!*~lN{8pK}Ez{2zJJTdqq5TKZFJAS5qA?3wfFo-JCofT0gEMyqU
zzN0UrI6vU+fm!IT?&Al_54T1fSrK!!)7>GMEiaKdtPst*$5I}D12uFa%!1}96kVZQ
zu&FK&dFw9K8<yhA(3AdeX?sFW;3ELi5+=ilYvV{9h3<S&5oFz@bnli5AILF@W=_es
zih4O~i3cx_2h_M#%|eSE9x^9lx?6$I2$o*zuPY@gp2|B#Q9HG5c62<Cz*GU>2{t#+
zpjU<!YsantuMAvT`x`py=P6dQr0zQo`Sg>F{XizH7Ve_1XRDZ$!%AR;zM;cTZ+FFx
zUjPvS00D^s9|RZy0ssjG0#A6tX#yJz0162Zk!6#Okp0~*EW-!@qBEV2G$GLOa{p?n
z9#ugmAlBIONUj4dyDteI1x_4VwY<=I6iOpo_VJ2|I`r&{YYVn=&Z~oGgxp(s28D#K
zMe+nz9q+c9_)ah`otuZ5O`(m`0Pk`Z7sCQf6^WhC{2DVai;i%@mo6`Z<e9Ec)L^z!
zJ^e`78-FA+JGR31L&WH)kPniYl(%?ajZDnB!yHF&%mIp3{$6glaSD791WsXiBU_0?
zxR4qtYZCpD0jP(W9#)J+fXV5Nu56V>rk@yKE8A1>u2VD9vs@=k8y;K>B*E1Ln$Ni-
n5>?)d^5cjU@!t24JXLnUVEu?)?E{fxrcsM-z{P+pGkx~<R=X+6

literal 0
HcmV?d00001

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
index 60da4b1..0eb26f1 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
@@ -10,7 +10,9 @@ Description = "Minimal debian root file system"
 LICENSE = "gpl-2.0"
 LIC_FILES_CHKSUM = "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe"
 FILESPATH_prepend := "${THISDIR}/files:"
-SRC_URI = "file://isar-apt.conf"
+SRC_URI = " \
+    file://isar-apt.conf \
+    file://raspbian.public.key.gpg"
 PV = "1.0"
 
 WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
@@ -61,10 +63,14 @@ do_bootstrap() {
        sudo umount -l "${ROOTFSDIR}/proc" || true
        sudo rm -rf "${ROOTFSDIR}"
     fi
+    if [ -n "${DISTRO_KEYRING}" ]; then
+        KEYRING_ARG="--keyring ${WORKDIR}/${DISTRO_KEYRING}"
+    fi
     E="${@bb.utils.export_proxies(d)}"
     sudo -E "${DEBOOTSTRAP}" --verbose \
                              --variant minbase \
                              --arch "${DISTRO_ARCH}" \
+                             ${KEYRING_ARG} \
                              ${@ gen_components_argument(d)} \
                              "${DISTRO_SUITE}" \
                              "${ROOTFSDIR}" \
@@ -125,4 +131,4 @@ do_deploy[dirs] = "${DEPLOY_DIR_IMAGE}"
 do_deploy() {
     ln -Tfsr "${ROOTFSDIR}" "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${DISTRO_ARCH}"
 }
-addtask deploy before do_build after do_apt_update
\ No newline at end of file
+addtask deploy before do_build after do_apt_update
-- 
2.13.6


We could, of course, also download the key and convert it to gpg format
on-the-fly for use by debootstrap.


The second problem is then apparently a bug related to qemu-debootstrap:
The second stage bootstrap is called without specifying the mirror URL
once again, and that causes debootstrap to leave DEF_MIRROR in
/etc/apt/sources.list behind. That's a debian mirror, not a raspbian,
and the succeeding installation will fail.

I worked around it via this:

diff --git a/meta-isar/conf/distro/raspbian-jessie.conf b/meta-isar/conf/distro/raspbian-jessie.conf
index 89b769f..36faf01 100644
--- a/meta-isar/conf/distro/raspbian-jessie.conf
+++ b/meta-isar/conf/distro/raspbian-jessie.conf
@@ -7,6 +7,7 @@ DISTRO_SUITE ?= "jessie"
 # For bootstrap purposes still needed:
 DISTRO_COMPONENTS ?= "main contrib non-free firmware"
 DISTRO_APT_SOURCE ?= "http://archive.raspbian.org/raspbian"
+DISTRO_APT_SOURCES += "conf/distro/raspbian-jessie.list"
 DISTRO_CONFIG_SCRIPT?= "raspbian-configscript.sh"
 DISTRO_KERNELS ?= "rpi rpi2 rpi-rpfv rpi2-rpfv"
 DISTRO_KEYRING ?= "raspbian.public.key.gpg"
diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta-isar/conf/distro/raspbian-jessie.list
new file mode 100644
index 0000000..c9a353c
--- /dev/null
+++ b/meta-isar/conf/distro/raspbian-jessie.list
@@ -0,0 +1 @@
+deb	http://archive.raspbian.org/raspbian jessie	main contrib non-free firmware
diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
index 0eb26f1..af9f494 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
@@ -75,6 +75,7 @@ do_bootstrap() {
                              "${DISTRO_SUITE}" \
                              "${ROOTFSDIR}" \
                              "${DISTRO_APT_SOURCE}"
+    sudo rm -f ${ROOTFSDIR}/etc/apt/sources.list
 }
 addtask bootstrap before do_build after do_unpack
 

Jan