From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6517599416265736192 X-Received: by 10.223.184.42 with SMTP id h39mr4339567wrf.4.1517574510340; Fri, 02 Feb 2018 04:28:30 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.165.85 with SMTP id j21ls588131wrb.12.gmail; Fri, 02 Feb 2018 04:28:29 -0800 (PST) X-Google-Smtp-Source: AH8x2241kpYAk2UopFZ+mNrniUOxekkPZgCD0qedQGWdzc75SLaoxwpzMThAK2SGpAVtqYsSIU2S X-Received: by 10.223.157.199 with SMTP id q7mr333891wre.11.1517574509823; Fri, 02 Feb 2018 04:28:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517574509; cv=none; d=google.com; s=arc-20160816; b=Xw7Zt+BHH9WGPS6n55fkYG+FzmoLIn/shkvEpe0LL8zcKxnjqi24rVIpbqJHu6OxUX Z/uOhbA+nRHe6NkicBFLALRDvxtWlmlstTQOtofgyGphmrDuK6ptcnaB988j8iNHZp/I 7JbYssZkPGPwUroOncssqqGZSj3GRTrFvsMf29s3x4+KM8V1V5aGh5XRxPJYZIwieMEw 0/1RGZZI0126QVTdmWYpWWGw7jvjLtujdu9zlBrK/IIA8LoEaiBDtW87rTrDwWXqX6r0 bE8wkSNdDqNHRlZxbLHRLkMGK4YFSRyDPf5EX3hm8MWibwROO6XtxiwgJe8Fe2tGAs3n vc/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=FK3VVqD0wCGm9NacUv+amYbxh8eMTB1NxqwcGSIXsIQ=; b=04Q/lpZbroUbndhbXgMl20p+4AkU0pCb2EGVWBXPqPmqzfw2vxTzymQNXy73wT+hGA yO/DVAv8B5rpyZtdPV7XH6BtgTogmtC3ozTbbFXgz9SAfVtSJ7MXbsiX47c5YPj2DWUR n35bjKrn8mJJ8600ZOM9fxV3kafHml0fJDqHn8RmIloWgF8mPgD8wdDQG7QFoWCcWJx8 6THJGAgH+Puulq9ukVzz3LkdkkWD5LjWjzXkZa8KD9ftxh0PqgeA7dYxxqkFiyMUxh9U G2ma/pjJzffButErf/Q8XKWM+jaUucVO6v75x62fe8gTo+geEEY5J1xraOjFDgu6WPzm /wog== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id i75si68999wmg.1.2018.02.02.04.28.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Feb 2018 04:28:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] (client.yota.ru [188.162.65.234] (may be forged)) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w12CSO46014975 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 2 Feb 2018 13:28:27 +0100 Subject: Re: Multi repo support To: Claudius Heine , "[ext] Claudius Heine" Cc: isar-users References: <7714f0e1-aaca-add2-eabc-738d4043c21c@siemens.com> <20180201161658.6b0af973@mmd1pvb1c.ad001.siemens.net> <20180201193458.24b6ac3f@mmd1pvb1c.ad001.siemens.net> <1517511108.2646.24.camel@denx.de> <1517519259.2646.39.camel@denx.de> <161535ba350.27ac.034a6b0541ed39b7fb4e17f4ac219eaa@ilbers.de> <1517553656.2646.52.camel@denx.de> From: Alexander Smirnov Message-ID: <23593adb-8fb1-f777-66bf-ced97dd26650@ilbers.de> Date: Fri, 2 Feb 2018 15:28:19 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <1517553656.2646.52.camel@denx.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: QCOm6sUCORXf Hi again! > Hi, > > On Fri, 2018-02-02 at 00:52 +0300, Alexander Smirnov wrote: >> Hi! >> >> Claudius Heine 2 февраля 2018 г. 0:07:49 написал: >> >>> Hi Alex, >>> >>> On Thu, 2018-02-01 at 23:47 +0300, Alexander Smirnov wrote: >>>> >>>> On 02/01/2018 09:51 PM, Claudius Heine wrote: >>>>> Hi, >>>>> >>>>> On Thu, 2018-02-01 at 19:34 +0100, Henning Schild wrote: >>>>>> Am Thu, 1 Feb 2018 16:16:58 +0100 >>>>>> schrieb "[ext] Henning Schild" : >>>>>> >>>>>>> Am Thu, 1 Feb 2018 15:54:26 +0100 >>>>>>> schrieb "[ext] Claudius Heine" >>>>>> com> >>>>>>> : >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I would like to start the discussion about how to best >>>>>>>> implement >>>>>>>> muti repository support in isar. >>>>>>>> >>>>>>>> Does someone already has some ideas or even something in >>>>>>>> the >>>>>>>> pipeline for this? >>>>>>>> >>>>>>>> If not then I do have an idea that was outlined together >>>>>>>> with >>>>>>>> Jan: >>>>>>>> >>>>>>>> Adding and configuring apt repositories should be done >>>>>>>> via >>>>>>>> config >>>>>>>> files. It should be possible to define own multiconfigs >>>>>>>> while >>>>>>>> including multiconfigs from other layers. These configs >>>>>>>> then >>>>>>>> append >>>>>>>> filepaths to a global variable. >>>>>>>> >>>>>>>> Every file that is added this way contains 'sources.list' >>>>>>>> compatible repository definitions. So one repo each line. >>>>>>>> >>>>>>>> For every line in those files a repository entry for >>>>>>>> multistrap.conf >>>>>>>> is created. Here we might need some more complex code to >>>>>>>> convert >>>>>>>> such a apt repo tripel to the right format multistrap >>>>>>>> expects. >>>>>>>> But >>>>>>>> by using the 'sources.lists' format we would be >>>>>>>> independent >>>>>>>> of >>>>>>>> multistrap and become more future proof. >>>>>> >>>>>> We will also need a way to tell apt the priorities of these >>>>>> repos. >>>>>> Multistrap just adds them and apt-get installs packages >>>>>> according >>>>>> to >>>>>> its default behavior. >>>>>> That means that a package with the same name and version will >>>>>> get >>>>>> picked from a "random" repo. Overlays will need to make sure >>>>>> to >>>>>> always >>>>>> have a greater version or we will need apt configuration. >>>>>> https://wiki.debian.org/AptPreferences >>>>>> The big question here is how/whether multistrap will handle >>>>>> apt >>>>>> preference. >>>>>> >>>>>> Alex already ran into that when he wanted to modify "hello". >>>>>> Renaming >>>>>> packages - like Alex suggested - is not the way to go. >>>>>> Because >>>>>> the >>>>>> ones >>>>>> we do overlay could be deps of packages we do not overlay. >>>> >>>> The version collision could be avoided by using epoch, and if I >>>> understand it correctly - this is the preferable way to assign >>>> apt >>>> priority. >>>> >>>> https://www.debian.org/doc/debian-policy/#s-f-version >>> >>> Where did you get that epoch is the preferable way to assign apt >>> priority? >> >> Not the apt, just specific package. Hmm, I'm a bit incorrectly called >> my >> pov. :-) I see epoch as an easy way to patch existing package from >> upstream. You don't care about upstream one's version even if it >> goes >> forward, until your epoch is higher - your package will be installed. >> Of >> course it doesn't mean that upstream epoch couldn't be changed too, >> but >> this happens quite rare. >> >>> >>> From you link: >>> >>> epoch >>> This is a single (generally small) unsigned integer. It may >>> be >>> omitted, in which case zero is assumed. If it is omitted then the >>> upstream_version may not contain any colons. >>> It is provided to allow mistakes in the version numbers of >>> older versions of a package, and also a package’s previous version >>> numbering schemes, to be left behind. >>> >>> I see epoch just as a possibility to fix mistakes or if upstream >>> changed their versioning scheme, not use it use it generally for >>> specifying package priorities. Using it that way sound more like a >>> hack >>> to me. Because what happens if debian raises the epoch of a >>> package? >>> Then you would have to raise yours again? Sounds like playing >>> poker. >> >> Could you please describe the case when you have two repositories >> which >> contain package with the same name-version but different content? > > With apt-preferences we could prefer every package from our own repo > with the same name AFAIK, but this would make the original package no > longer install-able beside that. > And that is a limitation of debian/dpkg and we can do nothing about it > in isar. This problem it not ours to solve. > > Ideally debian would introduce package namespaces or similar. But that > is completely out of isars scope. On our side we could only try some > hacks to avoid name collision, but I would rather spend time solving > other problems in isar than that, since just choosing a different name > is pretty easy. Multi-repo or reproducible builds are much more > important issues than trying to play cat and mouse with package names. > I see this. But what is the practical usecase to have the same packages in several repositories? Alex >>> Telling apt to prefer packages with specific versions or from >>> specific >>> repositories via apt-preferences sounds more like the right tool >>> for >>> the job. >>> >>> Cheers, >>> Claudius >>> >>>> >>>> Regarding renaming 'hello' there was a bit different case, our >>>> 'hello' >>>> has nothing common with the upstream one, moreover I've already >>>> modified >>>> it to use 'libhello'. So this is not the case when we patch the >>>> upstream >>>> package. So I decided to rename it, to avoid confusion. >>>> >>>> In case of patching the upstream package, we could just add epoch >>>> to >>>> version, so our package will be always in first prio. >>>> >>>> Also I'm not sure that when there are 2 packages with the same >>>> names >>>> and >>>> versions available in source.list - is a good practice. In this >>>> case >>>> you >>>> have no way to easily check if your rootfs contains correct >>>> package, >>>> 'dpkg -l' won't be enough. >>>> >>>> Alex >>>> >>>>> Thanks, yes that is a very good point. >>>>> >>>>> I would prefer the apt preferences settings. Maybe handle it >>>>> similar to >>>>> how I proposed the multi-repo support. A global variable where >>>>> apt >>>>> preference file paths are appended. >>>>> >>>>> If multistrap cannot support them with a reasonable complex >>>>> conversion >>>>> script then it might be time to say multistrap goodbye. >>>>> >>>>> Claudius >>>>> >>> >>> -- >>> DENX Software Engineering GmbH, Managing Director: Wolfgang >>> Denk >>> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, >>> Germany >>> Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx. >>> de >>> >>> PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 >>> B153 >>> Keyserver: hkp://pool.sks- >>> keyservers.net >> >> -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov