From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6982479695616933888 X-Received: by 2002:ac8:4799:: with SMTP id k25mr13806666qtq.333.1625773284341; Thu, 08 Jul 2021 12:41:24 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac8:4111:: with SMTP id q17ls1520057qtl.8.gmail; Thu, 08 Jul 2021 12:41:24 -0700 (PDT) X-Received: by 2002:ac8:549:: with SMTP id c9mr29531949qth.80.1625773283991; Thu, 08 Jul 2021 12:41:23 -0700 (PDT) Received: by 2002:a37:6d7:0:b029:3b3:3e2f:eba7 with SMTP id af79cd13be357-3b7d5282520ms85a; Thu, 8 Jul 2021 12:34:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwxzd9/52zMgJA+Of8B/RRqCRrE+EA1SoCM50/YdglJoOn2xxAxOz/Ja+JvYq+umfgUL0OU X-Received: by 2002:a1c:38c7:: with SMTP id f190mr35165634wma.30.1625772880692; Thu, 08 Jul 2021 12:34:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1625772880; cv=none; d=google.com; s=arc-20160816; b=F58oHx3v1yBU8k7+dADuaegE2qpYvVPquD+ATCsrHaBRd3ngTIHbvdmrUXmc6af9IW mt98dw5qGusUvHGvKlU9JTvOw5DQVP4wdknYoI66yGjno3sqEIHTd8tVE3dTFpZeQuGf uijDA+9+z+QB0u6BbjBMi11SDjpa4qLL/EXiIbr0pS6ekDlWO7aKcxFH7ky1nDJjB+6o DcKUR703uMozFYjsOVCjwGIJ2vrprvTiF1cf36TjM8UhfE9akV4KHE1zqsf/Iu5h0BWa USLc4dm3+d+GVhxfak0EfGRCiiMWPGHK30Ceh63vL2FKArgVlENz6L6BQelGxNfTmMbh KUHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date; bh=HQSdfEtTDUZ6GbX7x8rGwfC/UmzzkEhIZCLffrtPBEs=; b=KRFXlQaomXxA4YNCu/t5/Qe6yu1SiIE0Yqu1NFROkfsJVe5Pp2MJqRW2jlBsQqurzz cfuwzPaBqYrqw62QGrqq8i8TrZvq6/n6Q7fcSYVIrWVcOJlOCave08dSrfpD9hs8ija8 77UfRwH1xlpDBNskBcQKgC7puof5WN7M2WpbTzWxlkVrAJl8wjI2qX0Nc9vRAM8D6c/w HS4xHsYP0wSl0uofyewvrb9sGlnqvMdKxjWpCgfcsDQF+S4Gf+cBiEd9fKjFvaujaDUR EPJSgdDDUPgEIPiCo6Sz+3L9hKyOUPeZKs/DXHHB/665k0bm9/QP7F/XLQdBPr86/ySo Fsrw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 136.243.71.142 is neither permitted nor denied by best guess record for domain of helmut@subdivi.de) smtp.mailfrom=helmut@subdivi.de Return-Path: Received: from isilmar-4.linta.de (isilmar-4.linta.de. [136.243.71.142]) by gmr-mx.google.com with ESMTPS id u21si170777wmm.2.2021.07.08.12.34.40 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Jul 2021 12:34:40 -0700 (PDT) Received-SPF: neutral (google.com: 136.243.71.142 is neither permitted nor denied by best guess record for domain of helmut@subdivi.de) client-ip=136.243.71.142; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 136.243.71.142 is neither permitted nor denied by best guess record for domain of helmut@subdivi.de) smtp.mailfrom=helmut@subdivi.de Received: from isilmar-4.linta.de (isilmar.linta [10.0.0.1]) by isilmar-4.linta.de (Postfix) with ESMTP id 07EA6200239; Thu, 8 Jul 2021 19:34:38 +0000 (UTC) Date: Thu, 8 Jul 2021 21:34:22 +0200 From: Helmut Grohne To: Jan Kiszka Cc: Cedric Hombourger , isar-users , Baurzhan Ismagulov , "MacDonald, Joe" Subject: Re: [RFC] using lightweight containers instead of chroot Message-ID: References: <11b6ea24-b31a-a417-bcd9-0b32c5abe308@mentor.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-TUID: tuISOgve9H51 Hi Jan, On Thu, Jul 08, 2021 at 07:16:05PM +0200, Jan Kiszka wrote: > Yeah, not cleanly separated: We need DPKG_ROOT to install build > dependencies unprivileged into a build environment, and then we need > something ideally unprivileged - e.g. namespaces - to run the build > inside that environment. Still wrong. DPKG_ROOT is not meant to install build dependencies. I doubt that it covers sufficient packages to do so any time soon. I also don't see any point in doing so when you can just chroot (unprivileged via user namespaces) into it and install them there. The reason to want DPKG_ROOT is to prepare the root filesystem for your embedded board. If that's not what you want to do, DPKG_ROOT is not your tool. > We don't want to reimplement if key requirements can be fulfilled with > Debian's own tools. One of them is having control over how build > dependencies are installed and from where (use cases: isar-apt, offline > builds). Another one would be the ability to patch an original package > after unpacking its sources. I think the goal should be making Debian's tools fulfil your key requirements. (Regardless of whether using an abstraction layer or not.) Can you elaborate how you want to control your build dependencies? It would seem to me that you could just patch debian/control and be done, no? What else do you need to control? Do you want to change the resolver? The "from where" part seems relatively easy to me. All of the build tools support one way or another to supply extra repositories. All you need to do here is ensure that your extra packages from extra repositories have a higher version than those in your base distribution. Why does your build tool need to support patching after unpacking sources? I'm using that approach in rebootstrap and I consider it a design mistake. You can just extract your source package outside the builder, bump the version, apply your patches, create a new source package and build that. Creating actual source packages saves you from a lot of trouble. > Why can't that also support a local folder with a apt repo? Spawning > servers is surely doable but can become fun (unless using containers...) > when running multiple build in parallel. I've considered that, but support for this is lacking at this point. Ideally, you'd want to bind mount the local folder into the chroot. When using user namespaces your mapped root user may be underprivileged to see that local folder already. sbuild does not provide such means (unless you modify the underlying schroot, which requires root). pbuilder has --bindmounts (but doesn't use a user namespace). mmdebstrap can only sync directories, which means copying them. > If that is not going to be rebuilt for every package (cached baseline, > some dependency versions for all built packages in a run) and could > possibly also be reused for other things (running the imaging), it could > be a building block to replace our buildchroots. If not, we would have > to continue feeding it with separately bootstrapped chroots. Where does that "not rebuilt" requirement come from? Is that for performance reasons? If yes, please benchmark it. I've locally compared build times (sbuild/pbuilder/mmdebstrap) and the difference is small (<10 seconds). After all, extracting a pbuilder (or sbuild) base.tgz is not immediate either and mmdebstrap is quite fast. > Most of the Isar builds (complete images, not just packages) are already > happening inside containers (kas-isar). Thus we rather need to be > careful not requiring any nesting that could become complex - at best. > At the same time, we want Isar to continue work on a plain Debian system. You cannot use user namespaces in secure docker containers anyway, but then when you already are inside a container, you can assume root privileges and use chroot normally. > The key question for us remains what value the abstraction can have for > Isar, today and on the long run. Switching builders is not a daily > business so far. Isar needs one, and that needs to plug well into the > image creation workflow. If interacting with mdbp can be simpler than > talking to sbuild directly, that would be a clear gain. If it's rather > more complex under the Isar constraints while we do not have the > compelling use case for switching builders, it is likely not the best > way forward. I think the value is not to be gained by you, but by Isar users. You already found your preferred builder and it is easy to stick to that. Others may have set up a different one already. The value gained from the abstraction is to be able to seamlessly use the builder of the user's choice. Another aspect is the need for avoiding nesting of virtualization technology. When building inside kas-isar, you cannot use user namespaces (because secure docker containers cannot). When building on a plain Debian system, you want to use user namespaces to be able to build unprivileged. At present, the only builder that allows switching between user namespaces and no user namespaces is sbuild. mdbp also gives that flexibility by allowing you to seamlessly swap builders. So if sbuild suits your need, then go for it rather than reinventing the wheel. I caution that using sbuild in a reproducible way is hard. People do change its defaults to their needs. You need to pass a lot of options to sbuild to avoid using user's defaults. That was one of the lessons I learned the hard way when writing mdbp. Helmut