From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6547633542169362432 X-Received: by 2002:a2e:7a06:: with SMTP id v6-v6mr1033773ljc.18.1525249175815; Wed, 02 May 2018 01:19:35 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:2a03:: with SMTP id q3-v6ls649626ljq.9.gmail; Wed, 02 May 2018 01:19:35 -0700 (PDT) X-Google-Smtp-Source: AB8JxZolA2vos3jy2JDx5Tz3Tte/QRlyuS6LNr+wYrDa/BujsCvdFpubb1gzW+ZeyPURLi6ETXcB X-Received: by 2002:a2e:854c:: with SMTP id u12-v6mr79537ljj.6.1525249175155; Wed, 02 May 2018 01:19:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525249175; cv=none; d=google.com; s=arc-20160816; b=jUNU0JySmfIkTNxgRbYUyE9KxvZenTm9Tp31BZWlqyBkl15NDWqBQ71sLkAoGNQqQ+ fBP5lTkvE3EaeY3EsNrnTPjbh2P9cgZyvPhBWEwuuJBEzJ/eUdQp8RN1NqIFTqJNdsZT hDbiuGm3i6E/Czmrf23WIDIuGYxRb1NLiKIiWVc+XRvOPZenVcPdsXahghAxxMCaR162 fJr5kRDlpnB8jpqFsXpfdoYMiyIC2aQIEl6atmejr6KJ2AI9t8nFjs/4o7EuIIwRbx0u zoEMN17Lgzr5oBCbd5qWAlXpOwGDGtlhmTb6lv+irbBGPspA14XSZqBkDqMI9PXH6+HD ABCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=wIkNuy85jPyzwwvee4TKJ3B4n6RwZhgNEbHhkeUe4r4=; b=jyHR3GDw+rISh6NBxcnEkW2Fqk8qW/bhNESpZ7M4AjiNTLY9gaCTm18GLSuOd5aOn8 APASIAULue4Q2WJ9N6on0Xfo3opBkl3gjeIq2lNaOiAjEmDiRXWcoVYspdnwGpMBId2S 2p6EOOpxuz46Ll7qGsdkd95hnCHi6vgcJ7hw4dviG2cpXTipsO5n3vKUaeAwWqpXMl2d muxQ+s8Z55e0lwkoPul2qCkn5C8IACdBkUvelnM65UWEC7uN+iJkx8rqa1qNK7Zku6I2 YPjC/OEFor65QFkXU404vQZELSWMdLBZWvIjakGaXRXOzVqwEYoDdtf6XyWnPtK/Ta36 MGMg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id j3-v6si493074lfb.1.2018.05.02.01.19.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 May 2018 01:19:35 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w428JYFD025829 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 2 May 2018 10:19:34 +0200 Received: from md1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40]) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w428JYcx029836; Wed, 2 May 2018 10:19:34 +0200 Date: Wed, 2 May 2018 10:19:33 +0200 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [RFC][PATCH 0/3] Experiments with .dsc backend Message-ID: <20180502101933.78da7849@md1pvb1c.ad001.siemens.net> In-Reply-To: References: <20180423132146.14743-1-asmirnov@ilbers.de> <20180423163616.6f038064@mmd1pvb1c.ad001.siemens.net> <52a4801e-ef31-bdc8-5a77-b2def3b90503@ilbers.de> <20180423180307.111a3f62@mmd1pvb1c.ad001.siemens.net> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 8kDAwJHeov2I Am Fri, 27 Apr 2018 18:46:17 +0300 schrieb Alexander Smirnov : > On 04/23/2018 07:03 PM, Henning Schild wrote: > > Am Mon, 23 Apr 2018 18:26:10 +0300 > > schrieb Alexander Smirnov : > > > >> On 04/23/2018 05:36 PM, Henning Schild wrote: > >>> Am Mon, 23 Apr 2018 16:21:43 +0300 > >>> schrieb Alexander Smirnov : > >>> > >>>> Hi everybody, > >>>> > >>>> as it was planned initially, one of the Isar feature is the > >>>> ability to rebuild original Debian packages with/without > >>>> customizations. Doing this manually in classical Yocto/OE style > >>>> requires manual setting of lots of variables and having a copy of > >>>> Debian patch (*.debian.tar.xz) for this package in your recipe's > >>>> folder. Maintenance of this system becomes very complicated. > >>>> > >>>> I performed a small research regarding bitbake capabilities and > >>>> found an interesting trick: during bitbake parsing stage (in > >>>> anonymous tasks) we could re-construct whole recipe (including > >>>> tasks and variable), and this snapshot will be stored in recipe's > >>>> dynamic context, that is used during pipeline execution. > >>>> > >>>> This series is just a PoC, lots of issues are still open, like: > >>>> - Patching upstream packages > >>>> - Automatic cross-recipe dependency detection to avoid > >>>> duplications in recipe and debian/control files > >>>> - etc... > >>>> > >>>> To test this series just run: > >>>> > >>>> $ bitbake hello > >>>> > >>>> After build, in hello workdir there will be a newly-baked > >>>> identical copy of upstream package :-) > >>> > >>> Interesting idea, but my personal feeling is that Isar already > >>> uses a lot of bitbake magic and we should maybe try to avoid > >>> introducing more where feasible. > >>> > >> > >> Bitbake was designed for Open Embedded, so it doesn't consider our > >> Isar/Debian usecases. And roughly speaking, bitbake usage in Isar > >> is a kind of hack at all. In such questions, I'd more prefer to > >> focus on the features we need and extend bitbake, instead of > >> applying Yocto philosophy to Isar. But this also is only my > >> personal feeling. > >>>> Feedbacks, ideas, proposals are welcome! > >>> > >>> So my feedback would be that the .dsc to recipe step should be put > >>> in a helper tool in the scripts directory. People that want to > >>> modifiy upstream packages would generate an isar-recipe with the > >>> tool once. Modifications in the form of patches could go on top. > >>> Or would this limit the envisioned feature in any way? > >> > >> IMHO this is undesired behavior: > >> - Extra step to generate new recipes > > > > Yes, but you can see it. > > For now I'd like to try keeping user interface as it is. In general, > generated files are hard to maintain, especially when you mix them > with repository content (generate them inside existing meta). If you > generate these files in a separate meta - then you need to add this > meta to build, it's also extra step. Moreover in this case the > default clear source tree is inconsistent, generated part is missed > by default. > > Anyway this approach could be an option if I'll get in stuck with > current implementation. > > > > >> - Extra step to add these recipes to build > > > > No, not really. You will have to mention your recipe somewhere if > > you want it in your image. What is does and how does not change > > that fact. > > > > BTW, did you try overloading something like an essential package? > > Think of something that is in buildchroot and needs to be updated > > in there. (libc in the worst case ...) > > Not yet, probably I posted here too early. This is only PoC for > automatic .dsc parsing. Application use-cases are in todo. Probably > it makes sense to come with them, so we will have real cases to > discuss :-) > > > > >> - Headache with maintenance: you touch some dsc -> you need to > >> regenerate new recipes etc... > > > > True, but major changes in the upstream package will probably make > > your patches not apply anymore. So i suspect headache here as well, > > the one thing you do with such a tool is story copies of possibly > > outdated dscs, which indeed is a bad idea. > > So the online recipe generation is probably the way to go. > > > > In case of any Debian update you anyway have to specify new .dsc > file, because the previous one will be removed (until you use local > mirror). So automatic support of customized upstream package.. It's a > kind of "I want to believe" :-( > > So, let me think a bit more about user interface and use-cases. > > >> - You should somehow/somewhere list of .dsc files > >> So this looks like a completely new ecosystem. > >> > >> .dsc is standard format to describe Debian source packages. So > >> anyway, to be Debian compliant Isar should support this format. > >> > >>> > >>> I also fear that this could complicate the reproducable build > >>> story. > >> > >> In general this doesn't affect reproducibility at all. The key > >> difference from typical recipe (build from tarball), that SRC_URI > >> is derived dynamically from .dsc file during parsing stage. The > >> remaining part of the pipeline stays untouched. > > > > It affects it because we will have to redirect every fetch to a > > local cache, and it introduces fetching. > > Sorry, could you please explain what you mean here? What is the > difference of calling do_fetch and call fetcher manually? At the moment there is no difference. But patches that introduce reproducability might modify do_fetch to handle caching centrally. Maybe it will be done in another way, so just something to keep in mind. Henning > Alex > > > > > Henning > > > >>> > >>> Henning > >>> > >>>> With best regards, > >>>> Alex > >>>> > >>>> Alexander Smirnov (3): > >>>> buildchroot: Include texinfo to base system > >>>> classes/dsc: Basic Debian .dsc backend implementation > >>>> recipes-debian: Add example recipe to test Debian .dsc > >>>> support > >>>> > >>>> meta-isar/recipes-debian/hello/hello.bb | 10 ++ > >>>> meta/classes/debian-dsc.bbclass | 112 > >>>> +++++++++++++++++++++++ > >>>> meta/recipes-devtools/buildchroot/buildchroot.bb | 3 +- 3 files > >>>> changed, 124 insertions(+), 1 deletion(-) create mode 100644 > >>>> meta-isar/recipes-debian/hello/hello.bb create mode 100644 > >>>> meta/classes/debian-dsc.bbclass > >>>> > >>> > >