From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6444767352800149504 X-Received: by 10.28.215.201 with SMTP id o192mr413937wmg.4.1500591949875; Thu, 20 Jul 2017 16:05:49 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.63.147 with SMTP id m141ls461276wma.0.canary-gmail; Thu, 20 Jul 2017 16:05:49 -0700 (PDT) X-Received: by 10.223.175.195 with SMTP id y3mr518797wrd.29.1500591949322; Thu, 20 Jul 2017 16:05:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1500591949; cv=none; d=google.com; s=arc-20160816; b=DejWtRzRNuYmGvWI+czhHFV/GXrRs4rR7dlwEWdJlEjoIyVBmeDEUcCrBPvSvqV4jD o2+0KgeMZJf4dUwot6D1/zQD9zMpaNVLWJHQSg/EqYFJq/eVgKtyrWNEoEBIrjFC0czQ L2VJXkcxRSA6W1uNr07Vl/uvGcRDnsnZU5eEZpJ26xflYwvAeyG1W5Zk2JK6bU/LvMXK GgJNoCxjeWj+ur2GC9t/e/oi5tawheU/zMjHjPL2RyafB6xvAr1VyzRQvk0FFrsB+bsy 9BA0sLC+YIi0MkrvTmL+6Sr6hvZnJX4/qjKXmc+Xc3O6ctfzZckBQNm2cEDBvrU56o3m cUxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:cc:to:from:date :arc-authentication-results; bh=O0k1pzZ32BJSZVvdXU9SaROAF61x+b2MK2pWi0Xlo2M=; b=e+dS+VbCzIeMBAc+Q+biCXQQyQISQ4wNNNqmTADZsDA2ff4kLuAO+iLb7mwR4S60N2 UElhNMdlxo9OZMzcsm/mOCQY46zkOK34F5fC1ysK1dXch7/pMl0DUjEzyOp8U6eqEZud QhfLKM8OkDC5IoUz29ixVnR0uvKm9sXJDNYKOrxeXq2VaIaj6TaUAvXo2dUtm7NhgWFB G9KbyEcPiGvqBt3Av208/jKvAi0scpKERJ6o1UQ/3Vl8a2diyMbpPVUwvqvPEg+JEhi6 x7sy69WgndMH/5Pw1DiR9b89lrUOW33VXTFaOJn4IfbK8WUOAK58cuqozG1/dvwJrWEU mgDA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 85.214.62.211 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id u70si569wmu.3.2017.07.20.16.05.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Jul 2017 16:05:49 -0700 (PDT) Received-SPF: neutral (google.com: 85.214.62.211 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 85.214.62.211 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Received: from yssyq.radix50.net (dslb-094-216-124-124.094.216.pools.vodafone-ip.de [94.216.124.124]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v6KN5aCO020378 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jul 2017 01:05:37 +0200 Received: from yssyq.radix50.net (localhost [127.0.0.1]) by yssyq.radix50.net (8.14.4/8.14.4/Debian-8) with ESMTP id v6KN5U9g028482 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 21 Jul 2017 01:05:30 +0200 Received: (from ibr@localhost) by yssyq.radix50.net (8.14.4/8.14.4/Submit) id v6KN5UGd028481; Fri, 21 Jul 2017 01:05:30 +0200 Date: Fri, 21 Jul 2017 01:05:30 +0200 From: Baurzhan Ismagulov To: isar-users@googlegroups.com Cc: Henning Schild , Alexander Smirnov , KOBAYASHI Yoshitake , Daniel Sangorrin , kazuhiro3.hayashi@toshiba.co.jp, jan.kiszka@siemens.com Subject: Re: [RFC] can we just extend openembedded to get Isar 2.0 Message-ID: <20170720230529.GB5040@yssyq.radix50.net> Mail-Followup-To: isar-users@googlegroups.com, Henning Schild , Alexander Smirnov , KOBAYASHI Yoshitake , Daniel Sangorrin , kazuhiro3.hayashi@toshiba.co.jp, jan.kiszka@siemens.com References: <20170720103037.794aab2e@md1em3qc> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-TUID: 1eIIE7Eqbvvb Hello, thanks for all the ideas. The short answer is, Isar has started from scratch as a rootfs generator from existing binary distro + self-built packages. In plain German ;) , OE is not necessary for this purpose. OE is a distro, and Isar is a builder. Of course, this is not that simple. You, and other people at various conferences, ask about the tools that you know and want to use, and this is exactly what we care about: Re-use mainstream tools and don't invent our own wheel. What we are doing now is exactly bringing the tools many people are familiar with into Debian(-based system) development. Where the modified tools are hosted (OE vs. Isar repos) is secondary; the primary issue is to make them useful for both worlds -- one has to do that also if one inherits from OE. Developing this same feature requires just the same effort compared to adapting the OE tools for Debian directly in OE repos. Our plan is to harmonize the tools for both use cases, submit the patches, and see what the OE community thinks about that. So, to answer your original question, I don't exclude the possibility of using Isar on top of OE if it would ever make sense. However, keeping the possibility of using plain Isar is important for commercial product builders. Rationale: The short answer above, plus a major disadvantage: OE is huge, and this luggage is a problem for CM and integrators (I had to maintain OE in ClearCase, and I'm glad I don't anymore). Regarding bitbake, I don't agree it's retrofitting. Bitbake is a great generic tool, perfectly suitable for the task. In fact, we build other commercial embedded systems with bitbake, without Isar or OE. Other tools like wic, runqemu, etc. are more OE-centric (mostly due to sysroot), but I'm glad to see that the required changes were quite manageable. The problem with Debian support in OE is that it is very undebian. Debian has its strengths, and there are reasons why Debian developers strongly dislike OE (see e.g. [1], and I've heard many such discussions). Deby / meta-debian follows the OE way of building Debian packages, and currently considers moving to the Debian way. The reason is, as soon as you oeize a package, you have to maintain it. That is why Isar does things the Debian way and aims at avoiding massive package modifications; OE doesn't really help here. A couple of comments below. On Thu, Jul 20, 2017 at 11:15:09AM +0200, Jan Kiszka wrote: > > But a lot of things are still missing. For true customization one would > > need a way to smuggle files into the rootfs, patch files in there, > > execute scripts in there. Ideally with collision protection and special > > protection for config-files etc. Customization is implemented in configscript.sh. Some more structure could be introduced, but we didn't need it till now, as the changes beyond custom debs were reasonable in size. > > Claudius already suggested building custom .debs in Isar to solve that. Yes. we built them where the effort was justified. > > Another thing i already came across is Image conversion .raw > > -> .qcow2/.vdi/.vmdk all that is missing in Isar. Same was true for the > > wic support until recently. As introduced above, Isar follows the Debian way to reuse as much as possible. That is why just basing on OE won't make the tools work for the Isar layer OOTB -- it will require the very adaptations we do now. > > People that are coming from Yocto and want to switch to Isar will never > > get their recipes to work, because all the utils-classes are missing in > > Isar and would need to be ported/pulled-in. This is what we ideally don't want to have. It is possible to build packages in a basic OE way, but the preferred way is to debianize all packages. We want to have stuff like clean, etc., but duplicating the whole OE functionality doesn't make sense and won't work, since Isar and OE have different base systems and toolchains. > > Isar needs sudo, and someone has to integrate libpseudo. This is on TODO. > > Isar still has several small issues with correct rebuilds after config > > changes etc. These issues are inherited from bitbake and its default setting of doing so. IIRC, there was a var to change that behavior, but I couldn't remember its name, have to dig the code. > > OE solved all those problems already AFAIK, not the rebuild one. > > but what it can not do is > > fetch .debs from a mirror and bundle them together with multistrap. OE > > most likely contains a lot of other usefull stuff that we might need > > eventually. You can copy the recipe from Isar, and OE will be able to multistrap. The problem is, you can't do anything useful with that afterwards due to the different base system and compiler. OE != Debian, just like Ubuntu != Debian. > > Does it really make sense for us to reinvent all those OE-features in > > Isar, or should we just got the other way around and put the Isar > > features into OE? The truth is, both ways are the same due to the reasons above (except where the modified tools are hosted, which doesn't matter), and we are already going that way. Doing on top of OE doesn't make the tools work OOTB. But it brings the OE distro luggage, which is not necessary for Isar use cases. > > At a first glance it looks like you have to teach OE to get debian > > packages and create a rootfs from them. Tell it to not compile anything > > at first. Probably much like the do_rootfs from Isar. We might get away > > with a layer on top of OE and maybe a few patches to OE that can maybe > > be mainlined. Or maybe the whole thing could become mainline OE some > > day. IIUC, not compiling while keeping the whole vars intact would make that a mutant behaving against the expectations of the users. Changing the vars would be creating a small Isar inside OE; I don't think this can be done in a layer alone. Regarding mainlining -- we want to start with the tools. I doubt the two paradigms (distro vs. builder) can / should be aligned into a single system. 1. https://lists.debian.org/debian-embedded/2016/04/msg00013.html With kind regards, Baurzhan.