From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6458972999552860160 X-Received: by 10.46.92.134 with SMTP id q128mr262019ljb.20.1503991763871; Tue, 29 Aug 2017 00:29:23 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.150.11 with SMTP id y11ls153750wmd.21.gmail; Tue, 29 Aug 2017 00:29:23 -0700 (PDT) X-Google-Smtp-Source: ADKCNb6cFGP36loxDSsnWHqhXP+Sx1ZTNo+Fs9LOZHDw4LbVqXp7DkMxrLjaQHBHmepFl0qrMBLs X-Received: by 10.28.15.195 with SMTP id 186mr98729wmp.2.1503991763212; Tue, 29 Aug 2017 00:29:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1503991763; cv=none; d=google.com; s=arc-20160816; b=QVwjtgUF4tcH3yRVG6mkEpggcitFKjK6Dywp0mOvEip89c64Ufkw62NqHfS3/Vvsvc L7Na5hWkrohnr6DiREe0XPwm/yaYenEeWEqYbhsrYLbOUnSu5aP8HLCLfApU9pQqxveg wv0gwVeGaE/pMljhXs2PkoF2zXyYNfeKghkHALf0IO8qVmjTg0Vvug0cqm8GRt3MLfIF MrAj+2pxjTUPPcAhp8oHOtocYw3y24PUEuyeGLUFhZO5ap+VFNL7ElDy5CBAS8udwswy DZAQTNtdTM0UbUnaZCUo79bAvzTmlY7GnfrtWk28llksifnQeIA9slyHVmIB8N/5Y4uG MAQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=0MH4cosl5hqbK+rJICmitqXzQ965MoobfLyaQxIQZik=; b=e9V8jt1wDp+hMnx5sF2ldMCDN3I7OM0UkNY/2aIQmXTZ8OYEDzKjeDeJLIEpzBqb9b e2BifAw9YGPWqjJOZ3Po1hH5z3C2M/T6h5PlUPv7lILwl+ndETLybngGtnWtHOby2A+f YUTW9jlTCJ5MhGNiUQD3pUMrgCZZ3WbeYjLDfJluMOB6geL1z/iGOfhMs/ueVp1PR1Sd VZz9oWysT/nDj+UuHMeXquFc4CC0jc7yoTshzH6Z9Sb6kblIUKGXg7sLuJp6MPZw+21Z LNq0ei7nPgxRV1WjAD4PuaO88J8RcHR2L4E5/pAzrTTTQmXeUyY/3hqLTnTzjWZGBVUG /hdw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id 192si87175wmy.2.2017.08.29.00.29.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 29 Aug 2017 00:29:23 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v7T7TKML013261 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 29 Aug 2017 09:29:22 +0200 Subject: Re: [PATCH 6/6] doc/technical_overview: Describe binary cache To: Henning Schild Cc: isar-users@googlegroups.com References: <20170827151339.12806-1-asmirnov@ilbers.de> <20170827151339.12806-7-asmirnov@ilbers.de> <20170828173630.7815bb3f@md1em3qc> From: Alexander Smirnov Message-ID: <4e347de8-7ee4-66e8-88d1-8d1813e6d0a2@ilbers.de> Date: Tue, 29 Aug 2017 10:29:15 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20170828173630.7815bb3f@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: owun2I5LDDW+ > This should be done along the way in every single patch, not afterwards. I don't think that it really should. The requirement to documentation is to describe current code state in repo. So if something has been changed in design - HEAD of branch should contain relevant documentation, what could be achieved by dedicate patch. Moreover this single patch explicitly describes the influence of whole series to current design. > > Henning > > Am Sun, 27 Aug 2017 18:13:39 +0300 > schrieb Alexander Smirnov : > >> Describe Isar binary cache feature. >> >> Signed-off-by: Alexander Smirnov >> --- >> doc/technical_overview.md | 59 >> +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 55 >> insertions(+), 4 deletions(-) >> >> diff --git a/doc/technical_overview.md b/doc/technical_overview.md >> index 20e08b4..b3265dc 100644 >> --- a/doc/technical_overview.md >> +++ b/doc/technical_overview.md >> @@ -100,6 +100,43 @@ Target filesystem lifecycle can be described as >> following: >> - According to the list of custom packages in bitbake recipes, the >> initial filesystem will be populated by successfully built packages. >> >> +## 2.5 Binary Cache >> + >> +As mentioned above, apart from fetching binary packages from >> upstream apt +repositories, Isar is able to build custom packages >> from sources. To simplify +further custom packages management, Isar >> implements binary caching. + >> +Binary cache is a typical apt repository created by Isar to store >> newly built +packages. This repository has mainly two goals: >> + >> + - Simplify image filesystem populating and distributing binary >> packages to >> + other systems. >> + >> + - Reuse these binaries in next builds to avoid re-building from the >> + scratch everytime. >> + >> +Current Isar tree provides default cache repository: `meta-isar-bin`. >> +Isar binary cache is described by the following files: >> + >> + - `meta-isar-bin/conf/layer.conf` >> + >> + - `meta-isar-bin/files/distributions.in` >> + >> +After system build is complete, this repo contains binary packages >> for all the +distros and architectures, requested in multiconfig. So >> this folder can be +published and used as apt repo for already >> installed systems. + >> +An important note, that Isar creates local repositories using >> dedicated Debian +tool reprepro. This tool takes care about: >> + >> + - Maintenance of multiple architectures and distros. >> + >> + - Internal packages database management. >> + >> + - Repo signatures management. >> + >> + - Requests about available packages in repo (name, version etc...). >> + >> # 3 Isar Internal Processes >> >> ## 3.1 General Overview >> @@ -174,10 +211,17 @@ contain debian folder. The build process is >> implemented in >> 3. Run dpkg-buildpackage >> >> -4. Task `do_install`: install successfully built packages >> +4. Further package processing depends on binary cache option: >> + >> + 1. Task `do_install_to_deploy`: install successfully built >> packages if >> + binary cache is disabled: >> `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to deploy directory >> `${DEPLOY_DIR_DEB}` >> >> + 2. Task `do_install_to_cache`: install successfully built >> packages to >> + binary cache repository: >> + `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to `${DIR_CACHE}` >> + >> ## 3.5 Populate Target Filesystem >> >> Each target image can be extended by custom packages listed in >> IMAGE_INSTALL @@ -185,11 +229,18 @@ variable. Task `do_populate` >> performs the following: >> 1. Parse IMAGE_INSTALL variable. >> >> -2. Find respective packages in `${DEPLOY_DIR_DEB}`. >> +2. Further steps depends in binary cache option. >> +If binary cache is disabled: >> + >> + 1. Find respective packages in `${DEPLOY_DIR_DEB}`. >> + >> + 2. Copy packages to deb folder in dedicated target filesystem. >> + >> + 3. Execute dpkg command in chroot for all the copied packages. >> >> -3. Copy them to deb folder in dedicated target filesystem. >> +If binary cache is enabled: >> >> -4. Execute dpkg command in chroot for all the copied packages. >> + 1. Call apt-get install in chroot in target image filesystem. >> >> ## 3.6 Generate Bootable Image >> >