Re: [RFC PATCH 0/3] Reproducible build

[Claudius Heine <claudius.heine.ext@siemens.com>]

[-- Attachment #1: Type: text/plain, Size: 4575 bytes --]

Hi,

On 2018-06-11 10:45, [ext] Claudius Heine wrote:
> Hi Maxim,
> 
> On 2018-06-07 10:08, Maxim Yu. Osipov wrote:
>> Hi Claudius,
>>
>> As far as I understood, 'apt-move' doesn't fit your requirements.
> 
> The documented functionality of apt-move would fit the requirements, but 
> since its no longer maintained and has bugs that makes it incompatible 
> with debootstrap, it cannot be used.
> 
>> Nevertheless, diagram illustrating the approach you propose would be 
>> be very helpful.
>>
>> As for diagram you attached to email below, honestly, it's rather 
>> difficult to understand it - too many arrows :(. It's worth to 
>> additionally describe the steps in text (and put corresponding numbers 
>> on the picture).
> 
> Yes I know its a bit overwhelming with all those arrows. It was much 
> worse before and this is the result of much simplifications.
> 
> But either way, since we cannot use apt-move I have to investigate what 
> is possible with reprepro and change the diagram accordingly.

reprepro does provide many nice features that might be interesting to 
use. For instance the 'gensnapshot' command. This caused me to further 
redesign this approach as the diagram attached shows.

I know this kind of diagram does not 100% the UML meaning, but I could 
not find a diagram type that really fits to what I want to show here.

This time I try to explain a bit more about what those arrows mean and 
how to read this diagram.

Dashed lines symbolizes dependencies between steps or 'packages' (in 
this case I mean those 'isar-bootstrap', 'buildchroot', 'recipes' and 
'image'. So if they are followed from the 'debootstrap download' in the 
'isar-bootstrap' package upwards, then this is the execution order. When 
the 'package import' component is reached the next 'package' in the 
dependency graph are started. This is 'buildchroot' or 'image'. Normally 
that is the buildchroot. Only if there are no recieps around that need a 
buildchroot it could be skipped. That means the last step to be executed 
is the 'finished' one.

The arrows going out from those database things, mean that packages are 
used from their. Those arrows have the same arrow head as the dependency 
arrows. In this diagram I made the lines of the arrows that are going 
out from 'local partial upstream mirror' a bit thicker to better 
differentiate them from the other ones.

The arrows that are going to 'to local mirror', 'to isar repository or 
'create local snapshot' nodes have a slightly different arrow head. 
These arrows mean that there are debian packages that are added to repos.

The basic idea it that after every step that installs or upgrades 
packages to the rootfs those packages are added to the local partial 
mirror. This can be done by first doing a download-only step adding it 
to the repo and then doing an install from the repo or the cache.

I also changed how the repo is build. The current idea it to have one 
repository with 2 components. One component for all upstream packages 
and one for all isar built packages. When the snapshot is generated in 
the end it will create one containing both components.

Maybe we should start adding the build time to all files in the deploy 
directory. This way we could add this to the name of the snapshot as 
well, so the association between those is made clear.

What do you think?

best regards,
Claudius

> 
> Claudius
> 
>>
>> Kind regards,
>> Maxim.
>>
>> On 06/05/2018 12:42 PM, Claudius Heine wrote:
>>> Hi,
>>>
>>> On 2018-06-04 18:05, [ext] Claudius Heine wrote:
>>>> I will try to post a small graphic about this soon.
>>>
>>> I attached the design Jan and me came up with yesterday.
>>>
>>> Hopefully its understandable enough even with all those arrows 
>>> pointing around. I tried to vary the arrow lines and heads to signify 
>>> dependencies/execution order, using of repositories and deploying 
>>> packages to the repositories.
>>>
>>> The main point about this is that it will work with apt preferences 
>>> repository pinning and that we will try to move all installed 
>>> packages to the local cache after every step.
>>>
>>> I highlighted the new components in the diagram as green and the 
>>> changed components as light green.
>>>
>>> If this is the way we want to go then I would try to get a RFC 
>>> patchset started.
>>>
>>> Cheers,
>>> Claudius
>>>
>>
>>
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

[-- Attachment #2: reproducible-build-design-planuml.svg --]
[-- Type: image/svg+xml, Size: 46994 bytes --]