Re: Reproducibility of builds

[Alexander Smirnov <asmirnov@ilbers.de>]

Hi Jan,

On 11/29/2017 10:02 PM, Jan Kiszka wrote:
> On 2017-11-29 19:53, Alexander Smirnov wrote:
>> Hi everybody,
>>
>> I've started working on this topic and here I'd like to share my vision.
>> At the moment I've implemented simple PoC in my branch
>> 'asmirnov/build_rep'.
>>
>> What it does:
>>
>> 1. There is new recipe: base-apt. It provides task which:
>>
>>   - Fetches packages from origin Debian apt to local folder using
>> deboostrap.
>>   - Put these packages via 'reprepro' to local repository called 'base-apt'.
>>
>> 2. Buildchroot uses 'base-apt' to generate rootfs.
>>
>> 3. Isar image uses 'base-apt' and 'isar' repos to generate rootfs.
>>
>>
>>
>> What are the key benefits of this approach:
>>
>> 1. Download session for upstream packages is performed in a single step.
>>
>> 2. You could use your local 'versioned' apt repository instead of
>> downloading origin packages.
>>
>> 3. Having local apt repository managed by 'reprepro' provides us
>> possibility to implement version pinning. Reprepro provides lots of
>> things like:
>>   - Get package name.
>>   - Get package version.
>>   - Remove specific package from repo.
>>   - Add single package to repo.
>>
>> So in general, if we have know which package version we want to have, we
>> need to get binary with this version and put it to 'base-apt'.
>>
> 
> But this encodes the versions of the packages to be used implicitly into
> their unique presence inside some local apt repo, no?
> 
> I would prefer a solution that stores the packages list with versions as
> well and only uses that list, when provided, independent of the repo
> content. That way we can throw all downloaded packages back into a
> single archive repo. Have one repo per project version will quickly
> explode storage-wise (or you need extra deduplication mechanisms).
> 
> That said, I'm fine with getting there in several steps, and this can be
> a valid first one.
> 

I got it.

Here I only mean that there are some tools that could help us in 
implementing specific logic. At the moment I don't have the final 
vision, but hope it will appear during experiments with this PoC.

My main wish is to avoid manual hacks with Debian artifacts and use 
generic tools as much as possible.

Alex

> Jan
> 
>>
>>
>> Which issues I see at the moment:
>>
>> 1. The key issue for me the list of packages for 'base-apt'. So before
>> 'base-apt' task is executed, we should prepare full list of packages
>> that will be used by:
>>   - buildchroot (BUILDCHROOT_PREINSTALL).
>>   - packages to build (their build deps).
>>   - image (IMAGE_PREINSTALL).
>>
>> So I have an idea how to implement this via special tasks, will push
>> patch for RFC, but if you have your own proposals, I'll be happy to
>> discuss them!
>>
>> Alex
>>