From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6916819860201668608 X-Received: by 2002:a19:230d:: with SMTP id j13mr2407635lfj.378.1610464283944; Tue, 12 Jan 2021 07:11:23 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:810b:: with SMTP id d11ls614929ljg.11.gmail; Tue, 12 Jan 2021 07:11:23 -0800 (PST) X-Google-Smtp-Source: ABdhPJwOlmkMAeSGjKsg489M9wWPYM5Be9aiAD0CNM1+HcZotd9bycDS72IbjLWuUuuqumY1nV6V X-Received: by 2002:a2e:8148:: with SMTP id t8mr2110892ljg.203.1610464282943; Tue, 12 Jan 2021 07:11:22 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1610464282; cv=none; d=google.com; s=arc-20160816; b=oSKLdZ6ph1YAaVqXeEs6ezc0x3x2kDGhRTLtcVVCHBbamh1owI8z54xegmKct0qFYx Gwj/XBV4cKntE8rM6m0q1xSxM572sEArWly2RfHtnvMifqEZ/zINn0xPcw2QmZrhloYZ ODKTVb6IEmjkJ8r3zBQnA176luREfrEL5s+XHJexTBk3MH3bF3mXItyMQ5IyD6vCm2X1 kSsupAMh1nWi75jssjoGV5KOkwQqSnkRb4lwaCK7w+H5Bkh/b+cCsESAQ2+ndV+XsiZ4 MjseaqhOZ0PRT9yu0aU997c1VAJzgcIh+hs+a0bNdp0bYxEJXal67knIFTZlTyeuIbJr P5ew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=l2CjoB2ROTF1vmQm/ikxjxE3R8XKg2lNcW1bt8OwYFc=; b=OuE+lsw3JBpCaghdmb5nHtTm+xLPXaEy7mHIkufiOU4txwuYy1Dvi72minvlo8cq8h chm/7ggz/JBzhX0NoOB6iTo6DUYnz620ke2F6FpKZ2nIb7gUHIbhRuLmLPZcN9xdQrex 6h/j6G+VWtjnA0faheu45+aaz9ojw2DyFKdIlEFw9M9ydAFRPac5eP2MD3PUlGlDVQv3 QxlE9ypWkqC+zuPcB4dZ5JU6Zf5yREfgtplLbJ/A2NMmwfLT9nIO53devqkvbnpNPSSd OE7DXEtGYcHN8bGO6MqyqaRNN5U8DkjtjYQ9yS1foYZ3TWgsvXg9pBArhnTqWxGfly4J BMKQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id a136si129531lfd.5.2021.01.12.07.11.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Jan 2021 07:11:22 -0800 (PST) Received-SPF: neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 10CFBMBm010012 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Tue, 12 Jan 2021 16:11:22 +0100 Received: from md1za8fc.ad001.siemens.net ([139.22.47.251]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 10CFBLH2031864; Tue, 12 Jan 2021 16:11:21 +0100 Date: Tue, 12 Jan 2021 16:11:20 +0100 From: Henning Schild To: Silvano Cirujano Cuesta Cc: Subject: Re: [RFC PATCH 2/2] docs: document usage of sdk container images Message-ID: <20210112161120.0a19d2bb@md1za8fc.ad001.siemens.net> In-Reply-To: <134bd608-0319-7649-52a4-a5e647740d50@siemens.com> References: <20210112103338.14712-1-silvano.cirujano-cuesta@siemens.com> <20210112103338.14712-3-silvano.cirujano-cuesta@siemens.com> <20210112124021.5a9f44ce@md1za8fc.ad001.siemens.net> <20210112131814.5a3cf85a@md1za8fc.ad001.siemens.net> <134bd608-0319-7649-52a4-a5e647740d50@siemens.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-TUID: uTKIBLF5sODd Am Tue, 12 Jan 2021 15:52:53 +0100 schrieb Silvano Cirujano Cuesta : > On 12/01/2021 13:18, Henning Schild wrote: > > Am Tue, 12 Jan 2021 13:03:18 +0100 > > schrieb Silvano Cirujano Cuesta > > :=20 > >> On 12/01/2021 12:40, Henning Schild wrote: =20 > >>> Am Tue, 12 Jan 2021 11:33:38 +0100 > >>> schrieb "[ext] Silvano Cirujano Cuesta" > >>> : > >>> =20 > >>>> Signed-off-by: Silvano Cirujano Cuesta > >>>> --- > >>>> doc/user_manual.md | 70 > >>>> ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 > >>>> insertions(+) > >>>> > >>>> diff --git a/doc/user_manual.md b/doc/user_manual.md > >>>> index a4f3d1d..ff779b1 100644 > >>>> --- a/doc/user_manual.md > >>>> +++ b/doc/user_manual.md > >>>> @@ -834,6 +834,76 @@ ii crossbuild-essential-armhf > >>>> 12.3 all Inf ~# > >>>> ``` > >>>> =20 > >>>> +## Create a "containerized" ISAR SDK root filesystem > >>>> + > >>>> +### Motivation > >>>> + > >>>> +Distributing and using the SDK root filesystem created following > >>>> the instructions in "[Create an ISAR SDK root > >>>> filesystem](#create-an-isar-sdk-root-filesystem)" becomes easier > >>>> using container images (at least for those using containers > >>>> anyway) +A "containerized" SDK adds to those advantages of a > >>>> normal SDK root filesystem the comfort of container images. + > >>>> +### Approach + +Create container image with SDK root filesystem > >>>> with installed cross-toolchain for target architecture and > >>>> ability to install already prebuilt target binary artifacts. > >>>> +Developer: > >>>> + - runs a container based on the resulting container image > >>>> mounting the source code to be built, > >>>> + - develops applications for target platform on the container > >>>> and > >>>> + - leaves the container getting the results on the mounted > >>>> directory. + > >>>> +### Solution > >>>> + > >>>> +User specifies the variable `SDK_FORMAT` providing a > >>>> space-separated list of SDK formats to generate. +Supported > >>>> formats are: > >>>> + - `tar`: is the default, is the non-containerized format that > >>>> results from following the instructions in "[Create an ISAR SDK > >>>> root filesystem](#create-an-isar-sdk-root-filesystem)" > >>>> + - `docker-archive`: an archive containing a Docker image that > >>>> can be imported with [`docker > >>>> import`](https://eur01.safelinks.protection.outlook.com/?url=3Dhttps= %3A%2F%2Fdocs.docker.com%2Fengine%2Freference%2Fcommandline%2Fimport%2F&= ;data=3D04%7C01%7Cde173c00-e982-4fda-8644-47edf4671d63%40ad011.siemens.com%= 7C6debb708331c4458499308d8b709c297%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7= C0%7C637460599822778290%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjo= iV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DiAJ9RTgmXSiZJhSl= qinclxLduxg1FXJYJ%2Bt8%2Beqc4g0%3D&reserved=3D0) > >>>> + - `docker-daemon`: (not supported from inside of a container) > >>>> resulting container image is made available on the local Docker > >>>> Daemon > >>>> + - `containers-storage`: (not supported from inside of a > >>>> container) resulting container image is made available to tools > >>>> using containers/storage back-end (e.g. Podman, CRIO, > >>>> buildah,...) > >>>> + - `oci-archive`: an archive containing an OCI image, mostly for > >>>> archiving as seed for any of the above formats + =20 > >>> Maybe a script to postprocess the one tarball we have would be a > >>> better option. I do not understand why skopeo can not be used > >>> inside docker or podman, but we _really_ should not motivate > >>> anyone to run isar plain. Otherwise i agree that a bitbake task > >>> is a good idea and much better than postprocessing outside of the > >>> build system. =20 > >> What do you mean by post-processing? I mean, a Docker image is > >> nothing else but a couple of JSON and TAR files using a fixed file > >> tree structure, with SHAs... But anyway writing something that does > >> it is like reinventing the wheel, that's what tools like Umoci, > >> Skopeo,... are for. =20 > > I mean some sort of script that needs to be called after bitbake. > > Not nice but can be done outside of a container. =20 >=20 > Hmmm, obviously, but seems to be so obvious for me isn't that obvious > :-) >=20 > The additions DON'T NEED to run in a container, but CAN be run in a > container (e.g. kas-container). >=20 > If running bitbake OUTSIDE of a container (e.g. VM), then the formats > "docker-daemon" and "containers-storage" should work like a charm. >=20 > If running bitbake INSIDE of a container (e.g. kas-container), then > the formats "docker-daemon" and "containers-storage" won't work. >=20 > I don't know if doc/user_manual.md is the right place for all the > clarifications that appear to be missing (this one and the dependency > on skopeo and umoci), but they are clearly needed. >=20 > > =20 > >> My first PoC was in fact just a shell script post-processing the > >> TAR file. Technically feasible, but it doesn't feel like a > >> comfortable workflow... =20 > > Doing that in bitbake is for sure the best solution, given it > > works. =20 > Of course, it works! ;-) > > =20 > >> The initial idea was from Jan Kiszka (I forgot to add a > >> "Suggested-by" to the commits) and after some discussion about the > >> convenience of integrating it, I agreed that the workflow feels > >> much better this way. > >> > >> Skopeo can be used inside of Docker or Podman, it's in fact what > >> I'm doing. What doesn't work is adding the container image to a > >> Docker Daemon (unless you pass the Docker socket to the container, > >> what I wouldn't do) or containers/storage system (mounting the > >> storage directory, what I wouldn't do). IMO the target users > >> should know how to handle docker-archives (I can add that "docker > >> load" has to be used). =20 > > Maybe i am just confused by the two "(not supported from inside of a > > container)". Does this mean one can not create the images, or not > > use ... or not both? > > The answer should probably made more clear in the docs. > > > > If creation works and only use fails, that is all fine. If creation > > inside a container does not work ... i would refrain from writing > > that code in bitbake and go post-process (extra shell script or > > only doc) with it. =20 >=20 > As mentioned above, running bitbake for building the SDK container > image can be run from inside and outside of a container. The only > difference is that running in a container the host shouldn't be > reachable and therefore the formats "docker-daemon" and > "containers-storage" shouldn't work. >=20 > If SDK_FORMAT =3D "tar", then only the currently existing format > (tarballed root filesystem) gets generated. If SDK_FORMAT =3D "tar > docker-archive", then you get both the old format and a docker image > archive that can be loaded on any Docker installation. >=20 > Possible scenarios (probably too much information for the > documentation): >=20 > - Project A has a build system that generates a SDK container running > bitbake without a container selecting SDK_FORMAT=3D"docker-daemon". > After finalizing pushes the SDK docker image to a container image > registry (docker push ...). Developers pull the SDK container image > from the container image registry (docker pull ...), start a > container based on the image and mounting the directory with the > source code, edit the source code on the host IDE and build in the > container with the provided SDK. >=20 > - Project B has a build system that generates a SDK container running > bitbake with a container selecting SDK_FORMAT=3D"tar docker-archive". > Since the project doesn't have any container image registry, it > distributes the container image using a network share. Team B1 do > have a container image registry and distributes the image internally > using it. BTW, without Team B1 project B wouldn't need anything but > "tar". >=20 > Clearer now? I think so. One bitbake call in a container will not work for most of what you propose. It all needs another bitbake call outside ... which i would call "post-processing". That is not nice, there is no need to even write that in bitbake, maybe even harm because it leads people to running isar native. Lets face it, its again containers causing problems with containers. I say add a few words to the documentation and let the container boys deal with that stuff, outside of isar. In a code-base far far away. But that is just my frustrated take from all the container crap that is popping up all over the place, causing more problems than it solves. Henning >=20 > =C2=A0 Silvano >=20 > > > > Henning > > =20 > >> =C2=A0 Silvano > >> =20 > >>> Henning > >>> =20 > >>>> +User manually triggers creation of SDK formats for his target > >>>> platform by launching the task `do_populate_sdk` for target > >>>> image, f.e. +`bitbake -c do_populate_sdk > >>>> mc:${MACHINE}-${DISTRO}:isar-image-base`. +Packages that should > >>>> be additionally installed into the SDK can be appended to > >>>> `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` > >>>> (self-built). + +The resulting SDK formats are archived into > >>>> `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_for= mat}.tar.xz` > >>>> (being `sdk_format` each one of the formats specified in > >>>> `SDK_FORMATS`). +The SDK container directory `/isar-apt` > >>>> contains a copy of isar-apt repo with locally prebuilt target > >>>> debian packages (for ). +One may get into an SDK > >>>> container and install required target packages with the help of > >>>> `apt-get install :` command. +The > >>>> directory with the source code to develop on should be mounted > >>>> on the container (with `--volume > >>>> :`) to be able to edit > >>>> files in the host with an IDE and build in the container. + +### > >>>> Example + > >>>> + - Make the SDK formats to generate available to the task > >>>> + > >>>> +For one-shot builds (use `local.conf` otherwise): > >>>> + > >>>> +``` > >>>> +export BB_ENV_EXTRAWHITE=3D"$BB_ENV_EXTRAWHITE SDK_FORMATS" > >>>> +export SDK_FORMATS=3D"docker-archive" > >>>> +``` > >>>> + > >>>> + - Trigger creation of SDK root filesystem > >>>> + > >>>> +``` > >>>> +bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base > >>>> +``` > >>>> + > >>>> + - Load the SDK container image into the Docker Daemon > >>>> + > >>>> +``` > >>>> +xzcat > >>>> build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archi= ve.tar.xz > >>>> | docker load +``` + > >>>> + - Run a container using the SDK container image (following > >>>> commands starting with `#~:` are to be run in the container) + > >>>> +``` > >>>> +docker run --rm -ti --volume "$(pwd):/build" > >>>> isar-sdk-buster-armhf:latest +``` > >>>> + > >>>> + - Check that cross toolchains are installed > >>>> + > >>>> +``` > >>>> +:~# dpkg -l | grep crossbuild-essential-armhf > >>>> +ii crossbuild-essential-armhf 12.3 > >>>> all Informational list of cross-build-essential packages +``` > >>>> + > >>>> ## Creation of local apt repo caching upstream Debian packages > >>>> =20 > >>>> ### Motivation =20 >=20