Re: Reproducibility of builds

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2017-11-30 09:04, Alexander Smirnov wrote:
> Hi Jan,
> 
> On 11/29/2017 10:02 PM, Jan Kiszka wrote:
>> On 2017-11-29 19:53, Alexander Smirnov wrote:
>>> Hi everybody,
>>>
>>> I've started working on this topic and here I'd like to share my vision.
>>> At the moment I've implemented simple PoC in my branch
>>> 'asmirnov/build_rep'.
>>>
>>> What it does:
>>>
>>> 1. There is new recipe: base-apt. It provides task which:
>>>
>>>   - Fetches packages from origin Debian apt to local folder using
>>> deboostrap.
>>>   - Put these packages via 'reprepro' to local repository called
>>> 'base-apt'.
>>>
>>> 2. Buildchroot uses 'base-apt' to generate rootfs.
>>>
>>> 3. Isar image uses 'base-apt' and 'isar' repos to generate rootfs.
>>>
>>>
>>>
>>> What are the key benefits of this approach:
>>>
>>> 1. Download session for upstream packages is performed in a single step.
>>>
>>> 2. You could use your local 'versioned' apt repository instead of
>>> downloading origin packages.
>>>
>>> 3. Having local apt repository managed by 'reprepro' provides us
>>> possibility to implement version pinning. Reprepro provides lots of
>>> things like:
>>>   - Get package name.
>>>   - Get package version.
>>>   - Remove specific package from repo.
>>>   - Add single package to repo.
>>>
>>> So in general, if we have know which package version we want to have, we
>>> need to get binary with this version and put it to 'base-apt'.
>>>
>>
>> But this encodes the versions of the packages to be used implicitly into
>> their unique presence inside some local apt repo, no?
>>
>> I would prefer a solution that stores the packages list with versions as
>> well and only uses that list, when provided, independent of the repo
>> content. That way we can throw all downloaded packages back into a
>> single archive repo. Have one repo per project version will quickly
>> explode storage-wise (or you need extra deduplication mechanisms).
>>
>> That said, I'm fine with getting there in several steps, and this can be
>> a valid first one.
>>
> 
> I got it.
> 
> Here I only mean that there are some tools that could help us in
> implementing specific logic. At the moment I don't have the final
> vision, but hope it will appear during experiments with this PoC.
> 
> My main wish is to avoid manual hacks with Debian artifacts and use
> generic tools as much as possible.
> 

I do agree.

Jan