From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6458972999552860160 X-Received: by 10.46.8.10 with SMTP id 10mr267178lji.29.1503994003310; Tue, 29 Aug 2017 01:06:43 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.5.133 with SMTP id 127ls167477ljf.36.gmail; Tue, 29 Aug 2017 01:06:42 -0700 (PDT) X-Google-Smtp-Source: ADKCNb7hdbubu9Mf68t91lGTtKbr0HZOxB6aEZBNcitNva1AK7f6vfvkPBMeWU88hIp9RhHWWHt2 X-Received: by 10.25.201.131 with SMTP id z125mr235876lff.37.1503994002871; Tue, 29 Aug 2017 01:06:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503994002; cv=none; d=google.com; s=arc-20160816; b=qSaBEPIq7sHP7Q2RG2T8c9pxe5W2G0Feu98XBZvzrjW+5Ueacil+Ve1Okw1jrZl+bo IYywFmip+HdbkSG91Rdn50PjrW4uLRwiCZed0krUfPLo030iJr9XnoWlIyIcbysl001X odbY0aTr46zkRVuw3S7m3kM1FlL2WDPaOCxNfTvxWWYiU7S6mYCf07FBndRm6xpZ646q gyVr28aaI/SRKATqHG4pGmr52zFcpC/StiEN7NaNHgOLpqCpXsdPsOjBd0vnxOjQPMfx cam3f1UGQoatOI6uLXG+zhSiU93w5wtJ8zBZA9TroTFaENUWJh9ElgHvSBY8/E1teJhV DGOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=vdCuVd2+3lHx1sEnOaFC1gNT0wu2pDEuVT3UwcC8V38=; b=NgTQEvubCwVg3oksRNevimJBVIdzigWblgHA1sj2qEuKWUJfIOQ0dIl86dBckfJbwI VqfBZ8ocMQGjnkH/fV3fYWuLwGro2CNy/SEcwJrsTePglVdCx75ndyHln8nsnOp1MyLv RxcWYwaC21lVDbdZsifId+Q2ecFyhfwGhE+XhokvKNYYypYw4ffXvrID9gIt3JvgygY3 JdSYM4CgTrWR9StlmBE9b9PrOVVgIQpCVzKIFNzCgM3ox85AfpKX7lsXhHMa+bK3nwGR qfhxC5wIhY5N8Wb5sune7SdPSaV6n8evO3h3DkGg7g6s0JHQEbelfCxLCeAUxVmetakX nNbQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id i90si135113wmh.5.2017.08.29.01.06.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Aug 2017 01:06:42 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id v7T86gmU020759 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 29 Aug 2017 10:06:42 +0200 Received: from md1em3qc ([139.25.68.40]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id v7T86gKI028182; Tue, 29 Aug 2017 10:06:42 +0200 Date: Tue, 29 Aug 2017 10:06:46 +0200 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH 6/6] doc/technical_overview: Describe binary cache Message-ID: <20170829100646.477d7a81@md1em3qc> In-Reply-To: <4e347de8-7ee4-66e8-88d1-8d1813e6d0a2@ilbers.de> References: <20170827151339.12806-1-asmirnov@ilbers.de> <20170827151339.12806-7-asmirnov@ilbers.de> <20170828173630.7815bb3f@md1em3qc> <4e347de8-7ee4-66e8-88d1-8d1813e6d0a2@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 5C/skVYaxoGS Am Tue, 29 Aug 2017 10:29:15 +0300 schrieb Alexander Smirnov : > > This should be done along the way in every single patch, not > > afterwards. > > I don't think that it really should. The requirement to documentation > is to describe current code state in repo. So if something has been > changed in design - HEAD of branch should contain relevant > documentation, what could be achieved by dedicate patch. Moreover > this single patch explicitly describes the influence of whole series > to current design. Every commit should leave the whole repo in a consistent state. If you change something and change the doc in a later commit you have inconsistent intermediate states. I already explained that to you in detail when we talked about the order in which the document should appear in the repo. You can argue that humans can deal with such inconsistencies, true. Say there was a change like that, in a language that a machine reads: + mount /foo + umount /foo Would you want those two lines in one or in two patches? If two are OK, should they be merge directly after another or can we put 10 more between them? I think the answer is clear, since you want CI to succeed on every commit. So you want every commit to be consistent in terms of programming language. And i am saying we should do the same for natural/human language. Henning > > > > Henning > > > > Am Sun, 27 Aug 2017 18:13:39 +0300 > > schrieb Alexander Smirnov : > > > >> Describe Isar binary cache feature. > >> > >> Signed-off-by: Alexander Smirnov > >> --- > >> doc/technical_overview.md | 59 > >> +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 > >> insertions(+), 4 deletions(-) > >> > >> diff --git a/doc/technical_overview.md b/doc/technical_overview.md > >> index 20e08b4..b3265dc 100644 > >> --- a/doc/technical_overview.md > >> +++ b/doc/technical_overview.md > >> @@ -100,6 +100,43 @@ Target filesystem lifecycle can be described > >> as following: > >> - According to the list of custom packages in bitbake recipes, > >> the initial filesystem will be populated by successfully built > >> packages. > >> +## 2.5 Binary Cache > >> + > >> +As mentioned above, apart from fetching binary packages from > >> upstream apt +repositories, Isar is able to build custom packages > >> from sources. To simplify +further custom packages management, Isar > >> implements binary caching. + > >> +Binary cache is a typical apt repository created by Isar to store > >> newly built +packages. This repository has mainly two goals: > >> + > >> + - Simplify image filesystem populating and distributing binary > >> packages to > >> + other systems. > >> + > >> + - Reuse these binaries in next builds to avoid re-building from > >> the > >> + scratch everytime. > >> + > >> +Current Isar tree provides default cache repository: > >> `meta-isar-bin`. +Isar binary cache is described by the following > >> files: + > >> + - `meta-isar-bin/conf/layer.conf` > >> + > >> + - `meta-isar-bin/files/distributions.in` > >> + > >> +After system build is complete, this repo contains binary packages > >> for all the +distros and architectures, requested in multiconfig. > >> So this folder can be +published and used as apt repo for already > >> installed systems. + > >> +An important note, that Isar creates local repositories using > >> dedicated Debian +tool reprepro. This tool takes care about: > >> + > >> + - Maintenance of multiple architectures and distros. > >> + > >> + - Internal packages database management. > >> + > >> + - Repo signatures management. > >> + > >> + - Requests about available packages in repo (name, version > >> etc...). + > >> # 3 Isar Internal Processes > >> > >> ## 3.1 General Overview > >> @@ -174,10 +211,17 @@ contain debian folder. The build process is > >> implemented in > >> 3. Run dpkg-buildpackage > >> > >> -4. Task `do_install`: install successfully built packages > >> +4. Further package processing depends on binary cache option: > >> + > >> + 1. Task `do_install_to_deploy`: install successfully built > >> packages if > >> + binary cache is disabled: > >> `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to deploy > >> directory `${DEPLOY_DIR_DEB}` > >> > >> + 2. Task `do_install_to_cache`: install successfully built > >> packages to > >> + binary cache repository: > >> + `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to `${DIR_CACHE}` > >> + > >> ## 3.5 Populate Target Filesystem > >> > >> Each target image can be extended by custom packages listed in > >> IMAGE_INSTALL @@ -185,11 +229,18 @@ variable. Task `do_populate` > >> performs the following: > >> 1. Parse IMAGE_INSTALL variable. > >> > >> -2. Find respective packages in `${DEPLOY_DIR_DEB}`. > >> +2. Further steps depends in binary cache option. > >> +If binary cache is disabled: > >> + > >> + 1. Find respective packages in `${DEPLOY_DIR_DEB}`. > >> + > >> + 2. Copy packages to deb folder in dedicated target filesystem. > >> + > >> + 3. Execute dpkg command in chroot for all the copied packages. > >> > >> -3. Copy them to deb folder in dedicated target filesystem. > >> +If binary cache is enabled: > >> > >> -4. Execute dpkg command in chroot for all the copied packages. > >> + 1. Call apt-get install in chroot in target image filesystem. > >> > >> ## 3.6 Generate Bootable Image > >> > >