Re: [PATCH 6/6] doc/technical_overview: Describe binary cache

[Henning Schild <henning.schild@siemens.com>]

Am Tue, 29 Aug 2017 10:29:15 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> > This should be done along the way in every single patch, not
> > afterwards.  
> 
> I don't think that it really should. The requirement to documentation
> is to describe current code state in repo. So if something has been
> changed in design - HEAD of branch should contain relevant
> documentation, what could be achieved by dedicate patch. Moreover
> this single patch explicitly describes the influence of whole series
> to current design.

Every commit should leave the whole repo in a consistent state. If you
change something and change the doc in a later commit you have
inconsistent intermediate states. I already explained that to you in
detail when we talked about the order in which the document should
appear in the repo.
You can argue that humans can deal with such inconsistencies, true.

Say there was a change like that, in a language that a machine reads:

+ mount /foo
+ umount /foo

Would you want those two lines in one or in two patches? If two are OK,
should they be merge directly after another or can we put 10 more
between them?
I think the answer is clear, since you want CI to succeed on every
commit. So you want every commit to be consistent in terms of
programming language. And i am saying we should do the same for
natural/human language.

Henning

> > 
> > Henning
> > 
> > Am Sun, 27 Aug 2017 18:13:39 +0300
> > schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> >   
> >> Describe Isar binary cache feature.
> >>
> >> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> >> ---
> >>   doc/technical_overview.md | 59
> >> +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55
> >> insertions(+), 4 deletions(-)
> >>
> >> diff --git a/doc/technical_overview.md b/doc/technical_overview.md
> >> index 20e08b4..b3265dc 100644
> >> --- a/doc/technical_overview.md
> >> +++ b/doc/technical_overview.md
> >> @@ -100,6 +100,43 @@ Target filesystem lifecycle can be described
> >> as following:
> >>    - According to the list of custom packages in bitbake recipes,
> >> the initial filesystem will be populated by successfully built
> >> packages. 
> >> +## 2.5 Binary Cache
> >> +
> >> +As mentioned above, apart from fetching binary packages from
> >> upstream apt +repositories, Isar is able to build custom packages
> >> from sources. To simplify +further custom packages management, Isar
> >> implements binary caching. +
> >> +Binary cache is a typical apt repository created by Isar to store
> >> newly built +packages. This repository has mainly two goals:
> >> +
> >> + - Simplify image filesystem populating and distributing binary
> >> packages to
> >> +   other systems.
> >> +
> >> + - Reuse these binaries in next builds to avoid re-building from
> >> the
> >> +   scratch everytime.
> >> +
> >> +Current Isar tree provides default cache repository:
> >> `meta-isar-bin`. +Isar binary cache is described by the following
> >> files: +
> >> + - `meta-isar-bin/conf/layer.conf`
> >> +
> >> + - `meta-isar-bin/files/distributions.in`
> >> +
> >> +After system build is complete, this repo contains binary packages
> >> for all the +distros and architectures, requested in multiconfig.
> >> So this folder can be +published and used as apt repo for already
> >> installed systems. +
> >> +An important note, that Isar creates local repositories using
> >> dedicated Debian +tool reprepro. This tool takes care about:
> >> +
> >> + - Maintenance of multiple architectures and distros.
> >> +
> >> + - Internal packages database management.
> >> +
> >> + - Repo signatures management.
> >> +
> >> + - Requests about available packages in repo (name, version
> >> etc...). +
> >>   # 3 Isar Internal Processes
> >>   
> >>   ## 3.1 General Overview
> >> @@ -174,10 +211,17 @@ contain debian folder. The build process is
> >> implemented in
> >>      3. Run dpkg-buildpackage
> >>   
> >> -4. Task `do_install`: install successfully built packages
> >> +4. Further package processing depends on binary cache option:
> >> +
> >> +   1. Task `do_install_to_deploy`: install successfully built
> >> packages if
> >> +   binary cache is disabled:
> >>      `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to deploy
> >> directory `${DEPLOY_DIR_DEB}`
> >>   
> >> +   2. Task `do_install_to_cache`: install successfully built
> >> packages to
> >> +   binary cache repository:
> >> +   `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to `${DIR_CACHE}`
> >> +
> >>   ## 3.5 Populate Target Filesystem
> >>   
> >>   Each target image can be extended by custom packages listed in
> >> IMAGE_INSTALL @@ -185,11 +229,18 @@ variable. Task `do_populate`
> >> performs the following:
> >>   1. Parse IMAGE_INSTALL variable.
> >>   
> >> -2. Find respective packages in `${DEPLOY_DIR_DEB}`.
> >> +2. Further steps depends in binary cache option.
> >> +If binary cache is disabled:
> >> +
> >> +   1. Find respective packages in `${DEPLOY_DIR_DEB}`.
> >> +
> >> +   2. Copy packages to deb folder in dedicated target filesystem.
> >> +
> >> +   3. Execute dpkg command in chroot for all the copied packages.
> >>   
> >> -3. Copy them to deb folder in dedicated target filesystem.
> >> +If binary cache is enabled:
> >>   
> >> -4. Execute dpkg command in chroot for all the copied packages.
> >> +   1. Call apt-get install in chroot in target image filesystem.
> >>   
> >>   ## 3.6 Generate Bootable Image
> >>     
> >