From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6916819860201668608 X-Received: by 2002:a2e:b5aa:: with SMTP id f10mr1959236ljn.489.1610453920146; Tue, 12 Jan 2021 04:18:40 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:5816:: with SMTP id m22ls524439ljb.4.gmail; Tue, 12 Jan 2021 04:18:39 -0800 (PST) X-Google-Smtp-Source: ABdhPJwsRyQm6IBm/6vundkgNDJwkhTG209MLaON176dPUhSCnvUzyS0dFOHCuC7APzFvTrGrA6J X-Received: by 2002:a2e:6109:: with SMTP id v9mr1802327ljb.431.1610453918959; Tue, 12 Jan 2021 04:18:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1610453918; cv=none; d=google.com; s=arc-20160816; b=QnFhjCNEEm2141VH4X6sVdcLEtyoiGcVm7Yb/VOLo/sdriLisLze463uYiGdAQK6Ew rvjrlpZW77wtTXzRNRznhfBu+ybGuitE+AJF1IctJB4excPW4NlEHhQ15iQuMJQB+JpO rLMM2dsaH7GD14pJC9iOeE6SdKu3YRpholrm0I+NKp6toVbqPQ+evNVvoap5QSY4TWtZ NH3rU9UZGBlTaQ+efYzSLsUjxYX2Cl8UIS2RlD4dJ7/CDR2EYs5//7DqJlauxp8ii2xI EgsNhpZcwXk8xdggFmZvuresg0iQBbTJysX+ir1p9tSSTQW363/zMQZPd0oLbaqdKNQ0 9xww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=fANJgGoIzOyRwWCg/uY/qXIgcT4ZuiEFz1cD8Zn3DUw=; b=N7jrvARuqzGpOCmfKwwkS4mjY+VmeknqcUgeLBOoJZh7USdY6uVLmjL/KOHfRPB6bO tSx+z/NMERqdoijs3A4kkvR5jPl+hoc1zTVDLFD6dPJ4gho8PurG39eakyUsqpSgZ447 iG1gyjJNDMj0iaYQNSeTLF36iqyUExdwdmF1bVrKWakwoeUL5NER/i4a3co76pCV5zv2 awo9LnxuSS0csbcczHy2iC7rnU3WZVbGhIs+ExWj2Ham+lLQHBkg80qLSvU9akCiIfP6 C4B7FeUPPMr3e/DjECIAFX2OYj7V4m2M7sWMaeMAcmQmE1TYGvkk4PnpSPSGCicbbPSy nbuw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id 207si100989lfm.0.2021.01.12.04.18.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 04:18:38 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 10CCIcdS004344 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 12 Jan 2021 13:18:38 +0100 Received: from md1za8fc.ad001.siemens.net ([139.22.47.251]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id 10CCIFNc008817; Tue, 12 Jan 2021 13:18:15 +0100 Date: Tue, 12 Jan 2021 13:18:14 +0100 From: Henning Schild To: Silvano Cirujano Cuesta Cc: Subject: Re: [RFC PATCH 2/2] docs: document usage of sdk container images Message-ID: <20210112131814.5a3cf85a@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20210112103338.14712-1-silvano.cirujano-cuesta@siemens.com> <20210112103338.14712-3-silvano.cirujano-cuesta@siemens.com> <20210112124021.5a9f44ce@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-TUID: BoWXpWk9u/Hs Am Tue, 12 Jan 2021 13:03:18 +0100 schrieb Silvano Cirujano Cuesta : > On 12/01/2021 12:40, Henning Schild wrote: > > Am Tue, 12 Jan 2021 11:33:38 +0100 > > schrieb "[ext] Silvano Cirujano Cuesta" > > : > > =20 > >> Signed-off-by: Silvano Cirujano Cuesta > >> --- > >> doc/user_manual.md | 70 > >> ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 > >> insertions(+) > >> > >> diff --git a/doc/user_manual.md b/doc/user_manual.md > >> index a4f3d1d..ff779b1 100644 > >> --- a/doc/user_manual.md > >> +++ b/doc/user_manual.md > >> @@ -834,6 +834,76 @@ ii crossbuild-essential-armhf 12.3 > >> all Inf ~# > >> ``` > >> =20 > >> +## Create a "containerized" ISAR SDK root filesystem > >> + > >> +### Motivation > >> + > >> +Distributing and using the SDK root filesystem created following > >> the instructions in "[Create an ISAR SDK root > >> filesystem](#create-an-isar-sdk-root-filesystem)" becomes easier > >> using container images (at least for those using containers anyway) > >> +A "containerized" SDK adds to those advantages of a normal SDK > >> root filesystem the comfort of container images. + +### Approach + > >> +Create container image with SDK root filesystem with installed > >> cross-toolchain for target architecture and ability to install > >> already prebuilt target binary artifacts. +Developer: > >> + - runs a container based on the resulting container image > >> mounting the source code to be built, > >> + - develops applications for target platform on the container and > >> + - leaves the container getting the results on the mounted > >> directory. + > >> +### Solution > >> + > >> +User specifies the variable `SDK_FORMAT` providing a > >> space-separated list of SDK formats to generate. +Supported > >> formats are: > >> + - `tar`: is the default, is the non-containerized format that > >> results from following the instructions in "[Create an ISAR SDK > >> root filesystem](#create-an-isar-sdk-root-filesystem)" > >> + - `docker-archive`: an archive containing a Docker image that can > >> be imported with [`docker > >> import`](https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3= A%2F%2Fdocs.docker.com%2Fengine%2Freference%2Fcommandline%2Fimport%2F&d= ata=3D04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%7C= d97fc3d66aed4a29eb8e08d8b6f20e68%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0= %7C637460498015543275%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV= 2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3D1Ug2lVy%2F0mmsVu3G= jTA6gYuOj9rp8gFNouuvUHkBKvQ%3D&reserved=3D0) > >> + - `docker-daemon`: (not supported from inside of a container) > >> resulting container image is made available on the local Docker > >> Daemon > >> + - `containers-storage`: (not supported from inside of a > >> container) resulting container image is made available to tools > >> using containers/storage back-end (e.g. Podman, CRIO, buildah,...) > >> + - `oci-archive`: an archive containing an OCI image, mostly for > >> archiving as seed for any of the above formats + =20 > > Maybe a script to postprocess the one tarball we have would be a > > better option. I do not understand why skopeo can not be used > > inside docker or podman, but we _really_ should not motivate anyone > > to run isar plain. Otherwise i agree that a bitbake task is a good > > idea and much better than postprocessing outside of the build > > system. =20 >=20 > What do you mean by post-processing? I mean, a Docker image is > nothing else but a couple of JSON and TAR files using a fixed file > tree structure, with SHAs... But anyway writing something that does > it is like reinventing the wheel, that's what tools like Umoci, > Skopeo,... are for. I mean some sort of script that needs to be called after bitbake. Not nice but can be done outside of a container. > My first PoC was in fact just a shell script post-processing the TAR > file. Technically feasible, but it doesn't feel like a comfortable > workflow... Doing that in bitbake is for sure the best solution, given it works. > The initial idea was from Jan Kiszka (I forgot to add a > "Suggested-by" to the commits) and after some discussion about the > convenience of integrating it, I agreed that the workflow feels much > better this way. >=20 > Skopeo can be used inside of Docker or Podman, it's in fact what I'm > doing. What doesn't work is adding the container image to a Docker > Daemon (unless you pass the Docker socket to the container, what I > wouldn't do) or containers/storage system (mounting the storage > directory, what I wouldn't do). IMO the target users should know how > to handle docker-archives (I can add that "docker load" has to be > used). Maybe i am just confused by the two "(not supported from inside of a container)". Does this mean one can not create the images, or not use ... or not both? The answer should probably made more clear in the docs. If creation works and only use fails, that is all fine. If creation inside a container does not work ... i would refrain from writing that code in bitbake and go post-process (extra shell script or only doc) with it. Henning > =C2=A0 Silvano >=20 > > > > Henning > > =20 > >> +User manually triggers creation of SDK formats for his target > >> platform by launching the task `do_populate_sdk` for target image, > >> f.e. +`bitbake -c do_populate_sdk > >> mc:${MACHINE}-${DISTRO}:isar-image-base`. +Packages that should be > >> additionally installed into the SDK can be appended to > >> `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` > >> (self-built). + +The resulting SDK formats are archived into > >> `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_forma= t}.tar.xz` > >> (being `sdk_format` each one of the formats specified in > >> `SDK_FORMATS`). +The SDK container directory `/isar-apt` contains a > >> copy of isar-apt repo with locally prebuilt target debian packages > >> (for ). +One may get into an SDK container and install > >> required target packages with the help of `apt-get install > >> :` command. +The directory with the > >> source code to develop on should be mounted on the container (with > >> `--volume :`) to be able to > >> edit files in the host with an IDE and build in the container. + > >> +### Example + > >> + - Make the SDK formats to generate available to the task > >> + > >> +For one-shot builds (use `local.conf` otherwise): > >> + > >> +``` > >> +export BB_ENV_EXTRAWHITE=3D"$BB_ENV_EXTRAWHITE SDK_FORMATS" > >> +export SDK_FORMATS=3D"docker-archive" > >> +``` > >> + > >> + - Trigger creation of SDK root filesystem > >> + > >> +``` > >> +bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base > >> +``` > >> + > >> + - Load the SDK container image into the Docker Daemon > >> + > >> +``` > >> +xzcat > >> build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archive= .tar.xz > >> | docker load +``` + > >> + - Run a container using the SDK container image (following > >> commands starting with `#~:` are to be run in the container) + > >> +``` > >> +docker run --rm -ti --volume "$(pwd):/build" > >> isar-sdk-buster-armhf:latest +``` > >> + > >> + - Check that cross toolchains are installed > >> + > >> +``` > >> +:~# dpkg -l | grep crossbuild-essential-armhf > >> +ii crossbuild-essential-armhf 12.3 > >> all Informational list of cross-build-essential packages +``` > >> + > >> ## Creation of local apt repo caching upstream Debian packages > >> =20 > >> ### Motivation =20 >=20