Re: [PATCH v4 2/8] Prioritize isar-apt repo over all others

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-13 09:45, Alexander Smirnov wrote:
> 
> 
> On 02/13/2018 11:22 AM, Jan Kiszka wrote:
>> On 2018-02-13 09:09, Alexander Smirnov wrote:
>>>
>>>
>>> On 02/13/2018 10:22 AM, Jan Kiszka wrote:
>>>> On 2018-02-12 19:51, Alexander Smirnov wrote:
>>>>> On 02/12/2018 08:00 PM, Jan Kiszka wrote:
>>>>>> On 2018-02-12 17:31, Alexander Smirnov wrote:
>>>>>>>
>>>>>>>
>>>>>>> On 02/12/2018 04:45 PM, Jan Kiszka wrote:
>>>>>>>> On 2018-02-12 14:09, Alexander Smirnov wrote:
>>>>>>>>> On 02/12/2018 01:27 PM, Jan Kiszka wrote:
>>>>>>>>>> On 2018-02-12 11:11, Jan Kiszka wrote:
>>>>>>>>>>> On 2018-02-12 11:08, Alexander Smirnov wrote:
>>>>>>>>>>>> On 02/12/2018 12:41 PM, Jan Kiszka wrote:
>>>>>>>>>>>>> On 2018-02-12 10:33, Alexander Smirnov wrote:
>>>>>>>>>>>>>> On 02/11/2018 06:25 PM, Jan Kiszka wrote:
>>>>>>>>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This ensures that we can override packages from upstream
>>>>>>>>>>>>>>> Debian or
>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>> external sources with our self-built versions. We achieve
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> for now
>>>>>>>>>>>>>>> by asking multistrap to drop a preferences file for the
>>>>>>>>>>>>>>> buildchroot so
>>>>>>>>>>>>>>> that dependency installations use the right priority. For
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> image
>>>>>>>>>>>>>>> build, this does not work because all packages are pull
>>>>>>>>>>>>>>> during the
>>>>>>>>>>>>>>> bootstrap. Therefore, we set aptdefaultrelease to isar to
>>>>>>>>>>>>>>> ensure
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> our repo gets the higher priority.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'm not sure that I completely understand this. What is the
>>>>>>>>>>>>>> use-case for
>>>>>>>>>>>>>> this? If you build your own copy of upstream package, this
>>>>>>>>>>>>>> prioritization could be resolved by specifying suffix to
>>>>>>>>>>>>>> version.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Simple example: custom kernel build of linux-cip
>>>>>>>>>>>>>
>>>>>>>>>>>>> This generates a package called linux-image-amd64 (for
>>>>>>>>>>>>> amd64 as
>>>>>>>>>>>>> target),
>>>>>>>>>>>>> and that may have a version < debian's kernel. Now, if we
>>>>>>>>>>>>> do not
>>>>>>>>>>>>> prioritize isar-apt over upstream, the upstream kernel is
>>>>>>>>>>>>> taken,
>>>>>>>>>>>>> even
>>>>>>>>>>>>> when you switched PREFERRED_PROVIDER (in fact, the build will
>>>>>>>>>>>>> break
>>>>>>>>>>>>> when
>>>>>>>>>>>>> that virtual package linux-cip is selected but can't be
>>>>>>>>>>>>> installed
>>>>>>>>>>>>> due to
>>>>>>>>>>>>> the version preference "newest").
>>>>>>>>>>>>
>>>>>>>>>>>> Ok, but why you can't name you kernel 'linux-cip-image-amd64'?
>>>>>>>>>>>> Having
>>>>>>>>>>>> the same name as with original Debian one could be confusing.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> That is mandatory, in order to write packages that depend on THE
>>>>>>>>>>> kernel,
>>>>>>>>>>> instead of "my-custom-kernel-in-version-4.4.112".
>>>>>>>>>>
>>>>>>>>>> To make it clearer, just look at the example-module case: The
>>>>>>>>>> recipe
>>>>>>>>>> can
>>>>>>>>>> build both against the distro as well as all custom kernels,
>>>>>>>>>> without
>>>>>>>>>> touching it. And that is a pattern not limited to kernel<->module
>>>>>>>>>> relationships.
>>>>>>>>>
>>>>>>>>> kernel<->module pair are built from the same source tree, so
>>>>>>>>> here no
>>>>>>>>> problem with ${PN}-${PV} substitution as dependency. Also I've
>>>>>>>>> parsed
>>>>>>>>> pure Debian packages, nobody depends from 'linux-image-*', so no
>>>>>>>>> correlation with upstream packages.
>>>>>>>>
>>>>>>>> No, they can be independent, just like in the example I submitted.
>>>>>>>> It's
>>>>>>>> an external kernel module vs. some kernel. Please re-read.
>>>>>>>>
>>>>>>>> For Debian, this specific problem does not exist because Debian
>>>>>>>> only
>>>>>>>> ships in-tree modules or those that are built on the target on
>>>>>>>> installation. But you will find the exact same pattern when
>>>>>>>> browsing
>>>>>>>> external projects that come with modules.
>>>>>>>
>>>>>>> Ok, I see that you define KERNEL_ARCH variable in machine config and
>>>>>>> this variable is used to bind kernel<->module build process.
>>>>>>> Sometimes
>>>>>>> KERNEL_ARCH has to be changed according to your kernel
>>>>>>> configuration.
>>>>>>> It's not enough to change preferred_provider from "linux-distro" to
>>>>>>> "linux-cip" if you want to use LPAE for your ARM kernel. You also
>>>>>>> need
>>>>>>> to change KERNEL_ARCH from "armmp" to "armmp-lpae" to be correct.
>>>>>>
>>>>>> OK, but this cannot be defined by the kernel recipe. It has to be
>>>>>> defined by the machine setting. A kernel recipe could then select a
>>>>>> config that matches or reject the KERNEL_ARCH. Currently, we do
>>>>>> nothing
>>>>>> in this direction, that is a valid point for future improvements.
>>>>>>
>>>>>>>
>>>>>>> How you see the overall procedure to switch to custom kernel? Why
>>>>>>
>>>>>> PREFERRED_PROVIDER_virtual/kernel = "my-nice-kernel"
>>>>>>
>>>>>> See also
>>>>>> https://github.com/siemens/jailhouse-images/blob/master/conf/multiconfig/qemuamd64-jailhouse.conf
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> KERNEL_ARCH can't be "cip-armmp" in this case?
>>>>>>
>>>>>> Because the KERNEL_ARCH remains amd64, armmp, or whatever you
>>>>>> configured. Changing the kernel source doesn't change that in any
>>>>>> way.
>>>>>
>>>>> It doesn't remain for all the cases. ARM debian provides two kernel
>>>>> images:
>>>>>
>>>>>    - linux-image-armmp
>>>>>    - linux-image-armmp-lpae
>>>>>
>>>>> Current Isar doesn't handle this, but if you build your kernel with
>>>>> LPAE, I assume you should change KERNEL_ARCH respectively, otherwise
>>>>> there will be some inaccuracy. So the same machine could have several
>>>>> KERNEL_ARCHs.
>>>>
>>>> In fact, one machine would have to have multiple configs, selecting the
>>>> kernel flavor. In OE terms, that would make different machines, though
>>>> targeting the same hardware.
>>>>
>>>>>
>>>>> Then, what should I write in KERNEL_ARCH if I add new machine,
>>>>> let's say
>>>>> Cubieboard?
>>>>>
>>>>> In general, KERNEL_ARCH is not exactly what it's used here for.
>>>>> According to the:
>>>>>
>>>>> https://kernel-handbook.alioth.debian.org/ch-packaging.html
>>>>>
>>>>> the linux image name looks like the following:
>>>>>
>>>>>    - linux-image[-featureset]-flavour
>>>>>
>>>>> Also chapter 3.3 stays:
>>>>>
>>>>> 8<--
>>>>> Each architecture usually has multiple flavours of the binary kernel
>>>>> images. Different flavours correspond to different kernel
>>>>> configuration
>>>>> files, used to build the binary images from the same kernel tree.
>>>>> 8<--
>>>>>
>>>>> So having KERNEL_ARCH as the constant is deviation from Debian flow.
>>>>>
>>>>> In addition:
>>>>>
>>>>> 8<--
>>>>> Again, multiple flavours of binary images may be built from the
>>>>> featureset tree. For example, the i386 architecture has a number of
>>>>> different flavours, such as 686 and 686-pae, built from the common
>>>>> Debian kernel source. It also contains the rt featureset. The source
>>>>> tree for building the kernels for each of these featuresets is
>>>>> obtained
>>>>> by applying additional patches to the Debian kernel source. It may be
>>>>> used to build the rt-686-pae binary image flavours.
>>>>> 8<--
>>>>>
>>>>> So in my opinion, CIP is a feature set.
>>>>>
>>>>> I don't really think, that replacing upstream Debian kernel package by
>>>>> custom one and claiming that is the same will make Debian community
>>>>> happy...
>>>>>
>>>>>
>>>>> Ok, I see that we have different vision on this topic, I could stay
>>>>> with
>>>>> this implementation for now, but in general in my opinion this looks
>>>>> like a big hack and should be reworked.
>>>>
>>>> Debian has a strong word here, though it complicates things
>>>> significantly. But let me look into expressing the kernel selection via
>>>> KERNEL_WHATEVER. It will mean that PREFERRED_PROVIDER can no longer be
>>>> used, though. And we will have to generate / parameterize the control
>>>> files of all module packages. Maybe that's Debian intention as well.
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>> For example armbian uses branch names and machines to distinguish
>>>>>>> images:
>>>>>>>
>>>>>>> https://www.armbian.com/kernel/
>>>>>>>
>>>>>>>     - linux-image-dev-sun8i
>>>>>>>     - linux-image-next-odroidc2
>>>>>>>
>>>>>>> So if define KERNEL_ARCH as ${ORIGIN}-${MACHINE}, you never will get
>>>>>>> conflicts with the upstream. Also this doesn't affect modules
>>>>>>> building
>>>>>>> at all.
>>>>>>
>>>>>> This is about conflict: We are *replacing* the distro kernel. But
>>>>>> when
>>>>>> doing this, we do not want to change all its consumers. Armbian
>>>>>> likely
>>>>>> never ran into the use case of having to design such a full
>>>>>> replacement,
>>>>>> also regarding dependencies (I do not see them shipping any
>>>>>> out-of-tree
>>>>>> module packages).
>>>>>>
>>>>>> And the same is true for all the non-kernel cases we will face when
>>>>>> replacing library-like packages. Please do not focus all your
>>>>>> thoughts
>>>>>> on the kernel. This issue is absolutely generic.
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> What are the other use-cases for this pattern? As I mentioned
>>>>>>>>> above
>>>>>>>>> with
>>>>>>>>> 'glibc' example, this is unclean approach, distro could become
>>>>>>>>> broken at
>>>>>>>>> anytime. This could become robust *only* after build
>>>>>>>>> reproducibility
>>>>>>>>> integration, when you cache the upstream content. But with
>>>>>>>>> build-rep you
>>>>>>>>> could manually manage the content of the cache, so prioritization
>>>>>>>>> is not
>>>>>>>>> needed, you could just drop duplications from the cache.
>>>>>>>>
>>>>>>>> Every lib-dependency is another pattern for the special
>>>>>>>> kernel/module
>>>>>>>> case: Every distro package that come with a dependencies that you
>>>>>>>> want
>>>>>>>> to replace with a custom build AND you have good reason to
>>>>>>>> downgrade
>>>>>>>> that package, then you want your own packages to have higher prior.
>>>>>>>> That's why this hard-coding is the right thing to do, in
>>>>>>>> general: If
>>>>>>>> you
>>>>>>>> are building a package that competes with upstream, there must be a
>>>>>>>> reason...
>>>>>>>>
>>>>>>>>>
>>>>>>>>> In general having packages with the same name in 'source.list' and
>>>>>>>>> overriding version policies in apt looks very dangerous for me.
>>>>>>>>>
>>>>>>>>> If the current series requires prioritization for kernel only, it
>>>>>>>>> would
>>>>>>>>> be nice to specify only this package in apt preferences.
>>>>>>>>
>>>>>>>> See above, the kernel is just the first case. There will be
>>>>>>>> more, I'm
>>>>>>>> absolutely sure.
>>>>>>>
>>>>>>> I've asked especially about these cases. :-( At the moment there is
>>>>>>> only
>>>>>>> kernel case, which could be solved without this prioritization.
>>>>>>
>>>>>> Use your imagination: enforced replacement of ANY package we rebuild,
>>>>>> but with a lower version. We rebuild packages for a reason. That
>>>>>> implies
>>>>>> we want them to be actually used. And that means our packages
>>>>>> require a
>>>>>> higher priority.
>>>>>>
>>>>>
>>>>> My imagination told me how it will be difficult to debug segfaults on
>>>>> the target, because I pinned old libA in Isar-apt, while upstream
>>>>> dependents already have been compiled according the new version of
>>>>> libA.
>>>>> It wouldn't be so funny if nobody had faced with such problems in the
>>>>> past...
>>>>
>>>> Again: One guy shooting himself into the foot by explicitly stating
>>>> this
>>>> incorrect setup doesn't mean all others users have to give up on this
>>>> valid and required default property of self-build packages. It
>>>> remains a
>>>> fact that self-built packages are there to customize and overwrite
>>>> upstream packages, independent of how the kernel will look like. That's
>>>> one key purpose of Isar.
>>>>
>>>
>>> I'm not against package overwriting. I mean that there is *not the only
>>> one way* to do this. Apt provides way to overwrite packages via
>>> versions.
>>>
>>> How I see the creation of robust derivatives:
>>>
>>> 1. Create local copy for upstream apt (let's name it base-apt) with
>>> packages you are going to use for your build.
>>> 2. Choose package you want to modify, let's say libA.
>>> 3. Create recipe for it and update version to be ">" than original one.
>>> 4. Work with this environment.
>>
>> This doesn't solve intentional downgrades, like the kernel example would
>> have been one.
>>
>>>
>>> This approach at least somehow guarantees that you will have
>>> reproducible build. The packages in your base-apt are built according to
>>> the libA version you've customized. So my opinion: user could overwrite
>>> only package in local mirrors, i.e. mirrors that could not be updated by
>>> 3rd parties.
>>>
>>>
>>> What could happen bad with pinning over upstream (libA is from
>>> upstream):
>>>
>>> 1. libA.h v1.0 had:
>>>
>>> #define BUFFER_SIZE 32
>>>
>>> 2. libA.h v1.1 has:
>>>
>>> #define BUFFER_SIZE 16
>>>
>>> 3. Upstream contains packages that were already rebuilt according to the
>>> libA v1.1
>>>
>>> 4. You are still using libA v1.0 derivative.
>>>
>>> 5. Buffer overrun could occur now in any libA consumer from the
>>> upstream.
>>>
>>> So, there is *no* robust way to overwrite package from live upstream,
>>> you are always in the risk.
>>
>> Of course, if you overwrite something in an incompatible way, you have
>> to ensure updating all dependencies as well. That is the mistake done in
>> your example.
>>
>>>
>>> I could imaging that some business units may require this apt
>>> prioritization for their specific product's needs, so lets have this
>>> feature in Isar. But I think it shouldn't be enabled by default. Also
>>> the user documentation for this feature should describe all the risks
>>> that user takes if this option is enabled.
>>
>> Let's see how flexible the repo and package pinning configuration will
>> become with Claudius approach, and then we may switch to an explicit per
>> package enabling as well. More intuitive is the default I propose.
>>
>>>
>>>> If I manage to model the kernel differently, I will probably be able to
>>>> drop this patch from /this/ series. But the topic will only come up
>>>> again when Claudius proposes his multistrap rework which will include
>>>> shipping aptpreferences as well - and prioritizing our own repo
>>>> properly.
>>>>
>>>
>>> As I said, I could stay with this now to move forward. So the things
>>> discussed above could be fixed later.
>>
>> User-visible changes should not be lightly merged. So this discussion is
>> valuable when it helps to come up with consistent and sustainable
>> interfaces. So far, the user was not yet enough in our focus, and that
>> is hopefully changing now.
>>
> 
> What are the next steps with this series?
> Anyway I want at least patches 1 and 3 from it.

Yes, feel free to pick them ahead of the rest! I will work on the others
and rebase on next when ready.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux