Re: [PATCH 2/2] centralize multistrap configuration generation

[Claudius Heine <ch@denx.de>]

[-- Attachment #1.1: Type: text/plain, Size: 7885 bytes --]

On 02/15/2018 02:39 PM, Alexander Smirnov wrote:
> 
> 
> On 02/15/2018 04:20 PM, Claudius Heine wrote:
>> On 02/15/2018 01:37 PM, Alexander Smirnov wrote:
>>>
>>> Claudius Heine <claudius.heine.ext@siemens.com> 15 февраля 2018 г.
>>> 15:08:50 написал:
>>>
>>>> Hi Baurzhan,
>>>>
>>>> On 02/15/2018 12:42 PM, Baurzhan Ismagulov wrote:
>>>>> On Wed, Feb 14, 2018 at 02:15:09PM +0100,
>>>>> claudius.heine.ext@siemens.com wrote:
>>>>>> From: Claudius Heine <ch@denx.de>
>>>>>>
>>>>>> Previous there was redundant code between the buildchroot and
>>>>>> isar-image-base recipe.
>>>>>>
>>>>>> This patch moves the generation of the multistrap configuration to a
>>>>>> central python script that can be parametrized by both recipes.
>>>>>>
>>>>>> Signed-off-by: Claudius Heine <ch@denx.de>
>>>>>> ---
>>>>>>   .../recipes-core/images/files/multistrap.conf.in   | 38 ----------
>>>>>>   meta/lib/isar/multistrap.py                        | 80
>>>>>> ++++++++++++++++++++++
>>>>>>   .../buildchroot/files/multistrap.conf.in           | 37 ----------
>>>>>
>>>>> Could you please post two final multistrap.confs you'd like to get
>>>>> with your
>>>>> changes?
>>>>
>>>> 'like to get'? I don't want to get any other multistrap.conf than
>>>> before. This is not a change of the multistrap.conf files, but just how
>>>> they are generated. The python configparser created ini-file format
>>>> only
>>>> puts spaces around the '=' characters, but the resulting deserialized
>>>> objects is the same.
>>>
>>> The difference that I see IMHO: now you need to feed multistrap by
>>> community-standardized input, you propose to add custom tool that
>>> eats proprietary input.
>>
>> I don't get this point. What do you mean by that?
> 
> You've introduced new language, like:
> 
> +MULTISTRAP_ISAR_SOURCE = "copy:///${DEPLOY_DIR_APT}/${DISTRO}"
> +MULTISTRAP_ISAR_INSTALL = "${IMAGE_INSTALL}"
> +MULTISTRAP_ISAR_SUITE = "${DEBDISTRONAME}"
> 
> So to have working multistrap config I *have* to write pre-config in
> this new language, what it's the simplification here, one set of options
> is replaced by another...

??? Thats not a 'new language' that just bitbake variables. I think I
don't get what you are saying. Thats the same stuff that previous to
this change happend in this big wall of text sed command. IMO that this
is much more to the point and therefore easier to handle than this big
search and replace list.

>>
>>>
>>>>
>>>> Ok, that is not 100% true, I changed the name of the isar repository
>>>> section. Now its called 'isar' in the image (previously 'Isar') and
>>>> buildchroot (previously 'isar-apt') multiconfig. But that is it. It has
>>>> no different effect outside of multistrap.
>>>>
>>>>> I'd like to understand the reasons for moving from templates to a
>>>>> magic
>>>>> generator.
>>>>>
>>>>> In my experience, magic is usually poorly documented (if at all)
>>>>> and very
>>>>> ununderstandable for users. Simple change of a line requires
>>>>> locating the
>>>>> generator, understanding what it does (they tend to be hairy with
>>>>> tons of
>>>>> conditions) and patching the upstream [Isar] code. Templates have
>>>>> their
>>>>> disadvantages, but are straightforward for users. In general, I'd
>>>>> like to keep
>>>>> the current UX and stick with templates, which is possible most of
>>>>> the time.
>>>>
>>>> You prefer duplicated code? Two slightly different templates for two
>>>> buildchroot and image recipe?
>>>>
>>>
>>> AFAIK the key idea is to have the only single multistrap that works
>>> with upstream repos. Other multistraps fetch packages from local
>>> cache. So after this is done, the problem with duplicacations in
>>> configs will go.
>>>
>>>> Also this patch is only an intermediate step of the way for multi-repo
>>>> support. IMO it has value on its own, since it centralized the
>>>> generation of the multistrap configuration.
>>>> The current kind of 'template' does not support creating repository
>>>> entries on the fly.
>>>
>>> What is the use case to have them "on the fly"?
>>
>> See: Also this patch is only an intermediate step of the way for
>> multi-repo support.
>>
>> Use-case for multi-repo: docker-ce repository available.
> 
> Sorry, don't know what is docker-ce. Could you please describe whole
> repositories layout, just to understand the topic?

I have the need to install docker community edition from the developers
repository in my image. Currently I do it by downloading the 'deb' file
from the repo manually and installing it, here I have to take care of
updating the version to the latest one. What would be much better if I
can just add this repository to the multistrap process and install the
most current version like that. For that I need the ability to add
additional repositories to the multistrap configuration from other layers.

I don't understand what you mean by 'whole repositories layout'. You
mean like a tree, but from what repository?

> 
>>
>>>
>>> Alex
>>>
>>> It might be possible to extent the multiconfig with
>>>> shell scripting, but I prefer using python for such advanced logic.
>>>
>>> I don't care about python or bash >
>> At least Baurzhan seems to care, because he doesn't like debootstrap
>> because of it being in shell.
> 
> Aghr, sent the mail on interrupted line, sorry. :-(
> 
> I meant I don't care whether python or bash is used, my concern what is
> the input for they. If you prefer to parse multistrap config with python
> instead of bash, I could understand this.

I don't understand you again. I don't parse the multistrap config. I
just generate one.

> But here you want to introduce another language for doing the same things.

Again, what other language? I don't remember writing or using any
language parse in this patch.

> So at least for me, it would be much easier to take some template for
> multistrap config (which syntax is well known and available in man page)
> and create the custom layout that I need.

If you prefer the template approach, then we might need to introduce a
new templating language, because only search and replace via sed doesn't
cut it for the multi-repo feature.

Since I know jinja [1] we could do it with this. But it still might
result in something more hacky.

[1] http://jinja.pocoo.org/

Also since Jan, already decreed the switch to debootstrap I will now
continue working on a patchset for this.

Cheers,
Claudius

> 
> Alex
> 
>>
>> Cheers,
>> Claudius
>>
>>>
>>>>
>>>> regards,
>>>> Claudius
>>>>
>>>> -- 
>>>> DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
>>>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
>>>> Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to the Google
>>>> Groups "isar-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to isar-users+unsubscribe@googlegroups.com.
>>>> To post to this group, send email to isar-users@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/isar-users/d58343b9-c284-81e7-f743-2672c1f5f8ef%40siemens.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>
>>
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

           PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
                             Keyserver: hkp://pool.sks-keyservers.net


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]