Re: Reproducibility of builds

[Claudius Heine <ch@denx.de>]

[-- Attachment #1.1: Type: text/plain, Size: 2856 bytes --]

Hi Alex,

On 11/29/2017 07:53 PM, Alexander Smirnov wrote:
> Hi everybody,
> 
> I've started working on this topic and here I'd like to share my vision.
> At the moment I've implemented simple PoC in my branch
> 'asmirnov/build_rep'.
> 
> What it does:
> 
> 1. There is new recipe: base-apt. It provides task which:
> 
>  - Fetches packages from origin Debian apt to local folder using
> deboostrap.
>  - Put these packages via 'reprepro' to local repository called 'base-apt'.
> 
> 2. Buildchroot uses 'base-apt' to generate rootfs.
> 
> 3. Isar image uses 'base-apt' and 'isar' repos to generate rootfs.
> 
> 
> 
> What are the key benefits of this approach:
> 
> 1. Download session for upstream packages is performed in a single step.
> 
> 2. You could use your local 'versioned' apt repository instead of
> downloading origin packages.
> 
> 3. Having local apt repository managed by 'reprepro' provides us
> possibility to implement version pinning. Reprepro provides lots of
> things like:
>  - Get package name.
>  - Get package version.
>  - Remove specific package from repo.
>  - Add single package to repo.
> 
> So in general, if we have know which package version we want to have, we
> need to get binary with this version and put it to 'base-apt'.
> 
> 
> 
> Which issues I see at the moment:
> 
> 1. The key issue for me the list of packages for 'base-apt'. So before
> 'base-apt' task is executed, we should prepare full list of packages
> that will be used by:
>  - buildchroot (BUILDCHROOT_PREINSTALL).
>  - packages to build (their build deps).
>  - image (IMAGE_PREINSTALL).

Maybe try to do this flexible, because it should be also possible for
example to generate lxc images that are deployed to the final target in
the same isar run.

Also as Jan said, deduplication of packages so maybe try to fetch those
packages in the DL_DIR first, so that rebuilding is possible without
internet access, no tmp_dir, a populated DL_DIR and a package+version
list of sorts. Then have a task that copies those packages into the
tmp_dir into a repo within the tmp_dir and install from there. This way
the DL_DIR would only contain one instance of every packages and the
repo in the tmp_dir only have a copy (or maybe even just a symlink).

Archiving the DL_DIR would in this case be enough to build different
sets of images.

If you can solve this, than this solution looks promising.

Cheers,
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

           PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
                             Keyserver: hkp://pool.sks-keyservers.net


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]