From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7032253102499561472 X-Received: by 2002:a2e:b0e4:: with SMTP id h4mr29904978ljl.117.1637909041238; Thu, 25 Nov 2021 22:44:01 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:a7c4:: with SMTP id x4ls777609ljp.8.gmail; Thu, 25 Nov 2021 22:44:00 -0800 (PST) X-Google-Smtp-Source: ABdhPJz/Y9NA8kuWRD8Z6MU74t2UyIHLHoxoO6JAZnMWW3Cy3R43AQss+LHAluRclpDmQURPnBi/ X-Received: by 2002:a2e:b01a:: with SMTP id y26mr28431956ljk.317.1637909040014; Thu, 25 Nov 2021 22:44:00 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1637909040; cv=none; d=google.com; s=arc-20160816; b=lHWL2KxMU39Nhfejx+to0irD9AU/EL3U2f+YgdmmEbOxZwgPpVHbpUf4x01jA0Ox8c /4d9nra3ZZaVCv15crPzyb8es99UYcfP9bDt0C9SIwIFaEYu3ayGHHXIBuYNawc9fX4o NhhcvjCCfC1MxoyM/IX2FhCS/BaBGcWtF6LGkRWmmiAD8MZTEePDGv+s1ELwavM0yxqm 6cxdotqXrzFA9iGmlZ5tlkxzVUBPOeDFkCNuCZDTi8rsG9DnFDdnRFMT/MENgxntGddz oXxpYRl5yMrPG9F0vKuoH0r17fmB4XQHiPzD4vlNbiH7curHgV06iRhEU9oOHq7sWa/6 s6VA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=q0TyZEe0tgMwEcRu0YaPVlm3fa/0LEMCwowiNZH2zTc=; b=GRoRBR245uOBPzIF4ruVBEdcjC/F8ttRXfDSsvUB6+pelPwRzu7FRgAEC5XTuUi6Bx +NECKe4O7pkKO44mhumKunOBU7NvvhwlMPJM8VYM8p1z1WpFsj4vBelq420P3xM5gV3X 57MQZXuA6JlpAbnccQv/F3HtG129Nw6bmVS857XOYEtaSa66WKgI80C1y0hYH0VtDybb PgaJSUoys9SOEZ6AtHTz4fpoStI0PCsgq+HpMH5C1R4YoBxQHiQQLgG96U5T76L7etFk 5sUa7Ufn79gm86+gyNAN2IN8Qm4rYezwjTEzEK+XMtNfzO8ZX9QFsXO+6WsjYtKCXy6k oMww== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id d8si470410lfv.13.2021.11.25.22.43.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Nov 2021 22:43:59 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 1AQ6hxhI032287 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 26 Nov 2021 07:43:59 +0100 Received: from [167.87.0.111] ([167.87.0.111]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 1AQ6hwhV007034; Fri, 26 Nov 2021 07:43:58 +0100 Subject: Re: [PATCH v2 00/24] Sbuild/Schroot migration To: Uladzimir Bely , isar-users@googlegroups.com References: <20211119121333.13805-1-ubely@ilbers.de> <2071114.yiUUSuA9gR@home> From: Jan Kiszka Message-ID: Date: Fri, 26 Nov 2021 07:43:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: <2071114.yiUUSuA9gR@home> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: UWctq8P/Z4Bo On 23.11.21 14:05, Uladzimir Bely wrote: > In the email from Friday, 19 Nov 2021 г. 15:13:09 +03 user Uladzimir Bely > wrote: >> This is a patchset showing how sbuild/schroot tools can be integrated >> into Isar build system. >> >> ... >> >> Current limitations: >> - there is an unsolved problem with building foreigh architectures >> using kas-docker. What exactly is the problem here? >> >> - qemuamd64 images are buildable in kas-docker, but some changes >> in kas are reqired (like installing additional packages and adding >> 'builder' user to 'sbuild group). I used the following changes to >> build ghcr.io/siemens/kas/kas-isar:2.6.2-sbuild image based on >> current ghcr.io/siemens/kas/kas-isar:2.6.2: >> https://github.com/WiseLord/kas/commit/5b7b02aa33 >> >> ... >> meta/recipes-kernel/linux-module/files/debian/{rules => rules.tmpl} (50%) > > Here I would like to add some info regarding running the patchset with `kas- > docker` and/or using Gitlab CI. > > Initially, the patchset was checked in Jenkins CI (that doesn't use docker) > and sent to the maillist then. But there are some docker-related issues solved > a bit later. So here I'll try to explain them (and this is to be added to user > docs in the following patchset version), so someone could play with sbuild > patchset. > > 1. Additional packages required. > > The patchset requires 'schroot' and 'sbuild' packages to be installed on the > host. Also, they both should have at least "bullseye" version, otherwise some > things won't work properly (like caching .deb via 'sbuild' or foreign > architectures support in 'schroot'). > > While current (ghcr.io/siemens/kas/kas-isar:2.6.2) image doesn't include the > required packages, I temporary prepared and upuloaded one based on it > (ghcr.io/wiselord/kas-isar:2.6.2-sbuild). Look at Dockerfile.isar.sbuild chunk > in the patch https://github.com/WiseLord/kas/commit/38f4f11f11 for details. > > Gitlab > > Gitlab uses .gitlab-ci.yml for setup. So, to use modified image in Gitlab CI > user should simply change the first line: > >> - image: ghcr.io/siemens/kas/kas-isar:latest >> + image: ghcr.io/wiselord/kas-isar:2.6.2-sbuild > > This is required until official kas-isar image have everything included. > > 2. User should be added to 'sbuild' group. > > In case of 'kas-docker' it happend to be a bit tricky, because 'builder' user > is created 'on the fly', when container is run. I'm not an expert in 'kas' so > I simply pathced container-entrypoint to add user to sbuild group. Look at > `container.entrypoint` chunk in the patch for details. > > 3. Overlayfs restrictions. > > Docker uses overlayfs to mount dockerimage rootfs. Schroot uses /var/lib/ > schroot/union/{overlay,underlay} directories to keep 'basic' image and > temporary layers on top of it. > > So we happen to have 'overlayfs over overlayfs' situation that is not > supported by overlayfs kernel driver. > > The solutionis to use an external volume for the /var/lib/schroot/union/. Look > at 'kas-container' chunk in the patch for details. > I've commented on some aspects of you kas patch. It generally looks feasible, just some smaller details should be improved. > Gitlab > > It seems there is no way to say gitlab using external volume via the > configuration file `.gitlab-ci.yml`. But it can be done by the following > changes in /etc/gitlab/runner/config.toml: > >> -volumes = ["/cache"] >> +volumes = ["/m/ws-10/schroot-10a/union:/var/lib/schroot/union", "/cache"] > > So, gitlab will use external ("/m/ws-10/schroot-10a/union") directory on host. > Also, there should be empty "overlay" and "underlay" directories created in > it. That's also not unrealistic, given that we already need to provide special runners for the purpose of granting privileges and allowing binfmt_misc. One target would be https://gitlab.com/cip-project/cip-testing/gitlab-cloud-ci, the backend for many of our gitlab CI runners that are Isar-compatible. Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux