From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6899501277566205952 X-Received: by 2002:adf:f146:: with SMTP id y6mr5585736wro.298.1606415323755; Thu, 26 Nov 2020 10:28:43 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:414c:: with SMTP id h12ls1363107wmm.2.gmail; Thu, 26 Nov 2020 10:28:42 -0800 (PST) X-Google-Smtp-Source: ABdhPJyu7WHuvXwS/VwVV4jgky+146k1KI7icF4WU9qs/DooM/eURb/2DBz7dzhrP5LKssMayAjK X-Received: by 2002:a1c:5f54:: with SMTP id t81mr4738670wmb.77.1606415322777; Thu, 26 Nov 2020 10:28:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606415322; cv=none; d=google.com; s=arc-20160816; b=pvkxXkmiY7hZBhxLKf60Bchpi/I5//EVYFZ/POLXpGjba+KtLrCJaqhEb79bQcMUxT /G3JZjH6LLji2nRKdWGp7PbxN8pXEWjE5wTKxo0tEbfwYfE5S5/ediBEb2S9vhWNe3yF uSorVlCOpCU+S3FoSkOqAkM5NUmgSgd04fcgjf//SRjnxOJFRU/bxdD7EckGLwup4KuT rcQoKA/KfsEitdpwXBHQIxmajKdMv0QvUfCmwo2XRS85K/4peMZP674WOjyCqh4WppYg HsJfOY0BLo7JIN0n00YIWMW1nXY7JwHpCPx9c4mA4Uu0HsZ5NqxDal6HJQ8QOOQttfF1 sZGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from:ironport-sdr :ironport-sdr; bh=XGX27MHT04mwZFR0v+FnYgQ+LyRhmQmFWq8xqFCxES8=; b=A/jKfRHOlT2N3QN6w/1pYXY8Sf/LtxvM40GG3kT7NS6BbXzjdRWpdJLGVObPicEn+6 tPk7ByE+0fM/UxmcTKU8DdwzOj+rRaWRhWRdAzt8gzNiHWYmFkJl0rd6uDi4xlqTxKn8 w+IArQJPO+x+bmORhBPEHCnje3Gz396Glb5xwHXfn/MHGGAYrK0P3NNPWa+vRixfQxI6 ckXlC4fAmyQnmGulRhD84JmQELwgUiUccBG+XqgJ48LK58wnIcbS4Gy5ttzkvPEPOAmE J6SKFerkA7sp0vccxstUoaKRJ53uw+G9na/jYLt2ZuHCKV9jN7ElKUBJTGPjCv+mGpg6 cHEQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Return-Path: Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com. [68.232.137.252]) by gmr-mx.google.com with ESMTPS id y187si226499wmd.1.2020.11.26.10.28.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Nov 2020 10:28:42 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) client-ip=68.232.137.252; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com IronPort-SDR: 5swUSITIKorOJF+HBbysUdkiYmnh54ly17rrxCkThMdgVrAoxmIYihlfq3sivNX1hRu6riexkM HlaEyeawsKBXU8oCA/Kg6rWJRybAKl782I5povjVnJ59JdHAlXUDJ+cOneP9sAtu49QbfGyj7J NRYOCtnbT6D8H0t3A5UedSp3aYRs9B/VdtDOII3ruMOdr75WBG8bt3zV3r4BHpZ5XMGPTtJEc4 q5xvx7Osgt5Rmv2zj8XYuqkTpmaeD2dN8sgSgixqJIfKLCRX7QSOVPBj4S1ys07BiJanp7uVzK CKw= X-IronPort-AV: E=Sophos;i="5.78,372,1599552000"; d="scan'208";a="55644663" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa4.mentor.iphmx.com with ESMTP; 26 Nov 2020 10:28:40 -0800 IronPort-SDR: 915xDJOGtunPUmdhvt+YXHYY/NSmgJOcvgjYLGV0L29CHZrz6+A6jooIIlHOZEHBgs/j1S1vj7 fkMgtMEMehXwfqWU3iPm7HI0s7Vm7hEso8sgbl46dlu/4PerhnfjmFADvS9uuZY8YGIT9D89zX A/rXx9DmN5dO42yvIMiAvFswRsWHsnDRpTwmylAnm+yltluLWG1rPMiLqbXPgF2+yMh5hIoU0w 2VKFdjVyRWOSbUo2mamUoA/2XjmMjSb4KfqTHrSArReZ9twje7lZ0ermPqXBI5iIMOQU9Wm2Gs 4jk= From: Vijai Kumar K To: CC: Vijai Kumar K Subject: [PATCH v10 00/14] Debsrc Caching Date: Thu, 26 Nov 2020 23:58:04 +0530 Message-ID: <20201126182818.7261-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain Return-Path: Vijaikumar_Kanagarajan@mentor.com X-ClientProxiedBy: svr-orw-mbx-04.mgc.mentorg.com (147.34.90.204) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-TUID: eswVcOkmX7/8 Changes in v10: - Rebased against the latest next - Fixed some typos CI job running here: http://ci.isar-build.org:8080/job/isar_vkk_devel/91/ Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc10 Changes in v9: - Address review comments from Baurzhan Changes in v8: - Fix documentation as per Jan comments. Changes in v7: - Document details in user manual. Changes in v6: - Rebase against latest next - Address review comments from Henning Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc6.3 Changes in v5: The major changes in this series are as below, - (P2) Handle cases where HOST_DISTRO!=DISTRO. This avoids mixing of debs from different distro which inturn helps the src caching logic. - (P5) Avoid downloading package from other builds. - (P8) Add a reprepro based sanity test to check if the repo contains the sources for all the debs. - (P9) Introduce a new variable BASE_REPO_FEATURES which provides means to enable or disable various base-apt features. (cache-deb-src for now) - Some fixes in (P12 & P13) Also, addressed some review comments from Henning. Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc5 Changes in v4: - Use = format instead of just using to download the right version of source package. Changes in v3: - Take care of non-existent downloads/deb-src directory. Changes in v2: - Introduced additional patch to cache deb src - Rebased on top of henning/staging4 tree Vijai Kumar K (14): rootfs: Make rootfs finalize a separate task deb-dl-dir: Cache host distro debs separately meta: cache deb srcs as part of postprocessing deb-dl-dir: Make debsrc_download faster deb-dl-dir: Download files only belonging to the current image deb-dl-dir: Factor out the mounting part deb-dl-dir: Fix skipping of removed files repository: Add a sanity test to check missing sources base-apt: Introduce BASE_REPO_FEATURES repository: Fix failures due to missing section scripts/ci_build.sh: Enable deb-src caching rootfs: Fix possible overwrite of existing resolv.conf rootfs: Handle failures when postprocess is rerun doc/user_manual.md: Document details about deb-src caching doc/user_manual.md | 30 +++++++++ meta-isar/conf/local.conf.sample | 4 ++ meta/classes/deb-dl-dir.bbclass | 61 +++++++++++++++++- meta/classes/dpkg-gbp.bbclass | 8 ++- meta/classes/dpkg.bbclass | 8 ++- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 43 ++++++++++++- meta/classes/repository.bbclass | 13 +++- meta/classes/rootfs.bbclass | 63 +++++++------------ .../isar-bootstrap/isar-bootstrap.inc | 12 +++- meta/recipes-devtools/base-apt/base-apt.bb | 3 + scripts/ci_build.sh | 1 + 12 files changed, 196 insertions(+), 54 deletions(-) -- 2.17.1