From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6458972999552860160 X-Received: by 10.25.17.231 with SMTP id 100mr85770lfr.28.1503934588814; Mon, 28 Aug 2017 08:36:28 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.167.208 with SMTP id q199ls144084wme.7.gmail; Mon, 28 Aug 2017 08:36:28 -0700 (PDT) X-Google-Smtp-Source: ADKCNb5mESLJ0f0mYcNVE1r38031+xSLf5bzThyruWR4+qOBMLoM3ViHdfm2b9JjEjkTob7hVXEG X-Received: by 10.28.30.137 with SMTP id e131mr76616wme.0.1503934588362; Mon, 28 Aug 2017 08:36:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503934588; cv=none; d=google.com; s=arc-20160816; b=WQoE24dOF88um5kNLrd2lsVQUq5GBUqusmuZgBOo87HmfHn3sqv9JCEutKLdq5FTip mizBH5CYMjrnX7d6AbpuG5mLJuFCQOH4MuZBoI0mjJS9wwfffzuIsb5TEiesDWztfhjx U9O7c9y34/nglRKOGImX678z6R+XDVXlrfDvdTL6lEHLqA3CT1MFwh6/rEk3sall6w16 KCjcv2DDNJYGjgvOb8M7AogwvQ4+hUKhVRqLn0t9SqFH+9596BPwsxWUy3cIPYlgmT78 8kM7YJDYAuArcwdvl/xTZOJkTRpu6gR4vyq9zk0Rjfn94tM/aOluTZsS4JkpW4FomYJQ IBdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=9l6sCyVSlJFWvXgQH8n0j9ja7WtGCnlz4jkVHlXkfbg=; b=alf7Hbuo3uoWzmmrqlvUTlSoHu2DCC63gYO6bwNaePZVj0tG5CNxBHPXsIdXDTDCI1 Fe5z8P92JZGECTuJ/2H26hw/gxpz4mHnP67CwL1DkeRaZyzwW49Gw1aJ7709cYhHK6GS 4aZjsLu3cPs0pDjXhe2kn9BYlc66lOWKOxSKTi4SBaX0c64lN2sipKsKBfQGriEHqq3i nHMi/BT+emzf0agJL5FxavqO9TupLs8rS7RHQuAUNunlF5AxAG5PYj4rYoZPhHw4Dr9D Dl3SvK2d7LoY3GtR36MbqMMDGyVa28xF1cvt0lFkxfcIISMIpy5lqU4uuPf/8eIuLT3C /t3g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id w5si356690wmw.4.2017.08.28.08.36.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Aug 2017 08:36:28 -0700 (PDT) Received-SPF: neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 192.35.17.2 is neither permitted nor denied by best guess record for domain of henning.schild@siemens.com) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id v7SFaScO032514 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 28 Aug 2017 17:36:28 +0200 Received: from md1em3qc ([139.25.68.40]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id v7SFaRbh005907; Mon, 28 Aug 2017 17:36:27 +0200 Date: Mon, 28 Aug 2017 17:36:30 +0200 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH 6/6] doc/technical_overview: Describe binary cache Message-ID: <20170828173630.7815bb3f@md1em3qc> In-Reply-To: <20170827151339.12806-7-asmirnov@ilbers.de> References: <20170827151339.12806-1-asmirnov@ilbers.de> <20170827151339.12806-7-asmirnov@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: HdzXDCoe+Q89 This should be done along the way in every single patch, not afterwards. Henning Am Sun, 27 Aug 2017 18:13:39 +0300 schrieb Alexander Smirnov : > Describe Isar binary cache feature. > > Signed-off-by: Alexander Smirnov > --- > doc/technical_overview.md | 59 > +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 > insertions(+), 4 deletions(-) > > diff --git a/doc/technical_overview.md b/doc/technical_overview.md > index 20e08b4..b3265dc 100644 > --- a/doc/technical_overview.md > +++ b/doc/technical_overview.md > @@ -100,6 +100,43 @@ Target filesystem lifecycle can be described as > following: > - According to the list of custom packages in bitbake recipes, the > initial filesystem will be populated by successfully built packages. > > +## 2.5 Binary Cache > + > +As mentioned above, apart from fetching binary packages from > upstream apt +repositories, Isar is able to build custom packages > from sources. To simplify +further custom packages management, Isar > implements binary caching. + > +Binary cache is a typical apt repository created by Isar to store > newly built +packages. This repository has mainly two goals: > + > + - Simplify image filesystem populating and distributing binary > packages to > + other systems. > + > + - Reuse these binaries in next builds to avoid re-building from the > + scratch everytime. > + > +Current Isar tree provides default cache repository: `meta-isar-bin`. > +Isar binary cache is described by the following files: > + > + - `meta-isar-bin/conf/layer.conf` > + > + - `meta-isar-bin/files/distributions.in` > + > +After system build is complete, this repo contains binary packages > for all the +distros and architectures, requested in multiconfig. So > this folder can be +published and used as apt repo for already > installed systems. + > +An important note, that Isar creates local repositories using > dedicated Debian +tool reprepro. This tool takes care about: > + > + - Maintenance of multiple architectures and distros. > + > + - Internal packages database management. > + > + - Repo signatures management. > + > + - Requests about available packages in repo (name, version etc...). > + > # 3 Isar Internal Processes > > ## 3.1 General Overview > @@ -174,10 +211,17 @@ contain debian folder. The build process is > implemented in > 3. Run dpkg-buildpackage > > -4. Task `do_install`: install successfully built packages > +4. Further package processing depends on binary cache option: > + > + 1. Task `do_install_to_deploy`: install successfully built > packages if > + binary cache is disabled: > `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to deploy directory > `${DEPLOY_DIR_DEB}` > > + 2. Task `do_install_to_cache`: install successfully built > packages to > + binary cache repository: > + `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to `${DIR_CACHE}` > + > ## 3.5 Populate Target Filesystem > > Each target image can be extended by custom packages listed in > IMAGE_INSTALL @@ -185,11 +229,18 @@ variable. Task `do_populate` > performs the following: > 1. Parse IMAGE_INSTALL variable. > > -2. Find respective packages in `${DEPLOY_DIR_DEB}`. > +2. Further steps depends in binary cache option. > +If binary cache is disabled: > + > + 1. Find respective packages in `${DEPLOY_DIR_DEB}`. > + > + 2. Copy packages to deb folder in dedicated target filesystem. > + > + 3. Execute dpkg command in chroot for all the copied packages. > > -3. Copy them to deb folder in dedicated target filesystem. > +If binary cache is enabled: > > -4. Execute dpkg command in chroot for all the copied packages. > + 1. Call apt-get install in chroot in target image filesystem. > > ## 3.6 Generate Bootable Image >