Re: [RFC][PATCH 0/3] Experiments with .dsc backend

[Alexander Smirnov <asmirnov@ilbers.de>]

On 04/23/2018 05:36 PM, Henning Schild wrote:
> Am Mon, 23 Apr 2018 16:21:43 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> Hi everybody,
>>
>> as it was planned initially, one of the Isar feature is the ability to
>> rebuild original Debian packages with/without customizations. Doing
>> this manually in classical Yocto/OE style requires manual setting of
>> lots of variables and having a copy of Debian patch (*.debian.tar.xz)
>> for this package in your recipe's folder. Maintenance of this system
>> becomes very complicated.
>>
>> I performed a small research regarding bitbake capabilities and found
>> an interesting trick: during bitbake parsing stage (in anonymous
>> tasks) we could re-construct whole recipe (including tasks and
>> variable), and this snapshot will be stored in recipe's dynamic
>> context, that is used during pipeline execution.
>>
>> This series is just a PoC, lots of issues are still open, like:
>>   - Patching upstream packages
>>   - Automatic cross-recipe dependency detection to avoid duplications
>> in recipe and debian/control files
>>   - etc...
>>
>> To test this series just run:
>>
>>   $ bitbake hello
>>
>> After build, in hello workdir there will be a newly-baked identical
>> copy of upstream package :-)
> 
> Interesting idea, but my personal feeling is that Isar already uses a
> lot of bitbake magic and we should maybe try to avoid introducing more
> where feasible.
> 

Bitbake was designed for Open Embedded, so it doesn't consider our 
Isar/Debian usecases. And roughly speaking, bitbake usage in Isar is a 
kind of hack at all. In such questions, I'd more prefer to focus on the 
features we need and extend bitbake, instead of applying Yocto 
philosophy to Isar. But this also is only my personal feeling.

>> Feedbacks, ideas, proposals are welcome!
> 
> So my feedback would be that the .dsc to recipe step should be put in a
> helper tool in the scripts directory. People that want to modifiy
> upstream packages would generate an isar-recipe with the tool once.
> Modifications in the form of patches could go on top.
> Or would this limit the envisioned feature in any way?

IMHO this is undesired behavior:
  - Extra step to generate new recipes
  - Extra step to add these recipes to build
  - Headache with maintenance: you touch some dsc -> you need to 
regenerate new recipes etc...
  - You should somehow/somewhere list of .dsc files
So this looks like a completely new ecosystem.

.dsc is standard format to describe Debian source packages. So anyway, 
to be Debian compliant Isar should support this format.

> 
> I also fear that this could complicate the reproducable build story.

In general this doesn't affect reproducibility at all. The key 
difference from typical recipe (build from tarball), that SRC_URI is 
derived dynamically from .dsc file during parsing stage. The remaining 
part of the pipeline stays untouched.

> 
> Henning
> 
>> With best regards,
>> Alex
>>
>> Alexander Smirnov (3):
>>    buildchroot: Include texinfo to base system
>>    classes/dsc: Basic Debian .dsc backend implementation
>>    recipes-debian: Add example recipe to test Debian .dsc support
>>
>>   meta-isar/recipes-debian/hello/hello.bb          |  10 ++
>>   meta/classes/debian-dsc.bbclass                  | 112
>> +++++++++++++++++++++++
>> meta/recipes-devtools/buildchroot/buildchroot.bb |   3 +- 3 files
>> changed, 124 insertions(+), 1 deletion(-) create mode 100644
>> meta-isar/recipes-debian/hello/hello.bb create mode 100644
>> meta/classes/debian-dsc.bbclass
>>
>