From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6899390751905415168 X-Received: by 2002:aa7:dc05:: with SMTP id b5mr2073826edu.47.1606389589506; Thu, 26 Nov 2020 03:19:49 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:d151:: with SMTP id r17ls1870309edo.1.gmail; Thu, 26 Nov 2020 03:19:48 -0800 (PST) X-Google-Smtp-Source: ABdhPJwjoMHn/pKKThQgUb1BnJpxSJhmAoejpoFhAZyf83j6eZr8ED45wIpJcwnhpEO7ji9DxbDv X-Received: by 2002:a05:6402:1813:: with SMTP id g19mr2075821edy.105.1606389588444; Thu, 26 Nov 2020 03:19:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606389588; cv=none; d=google.com; s=arc-20160816; b=W6RLfclqz1Cy89A2IxYbn/uzfxPpPeRfPhv3BveczYAupNqHRrfRQI/AeMpW+6JdFs sO7oEaeYFl2acPu2w4dSveTM5v4OBhdXvCPBKTQJMZUJdTsT4tHOC1ipvSIAG2VWdWra GUZ+0nZgkjcMdSHXpRukJdWc3+KcNhwZaByALHvZsdBodEPKWAl81SC2IcUAdhAnkBs4 9ssJhn/d/yihKkTRKaPPUrlJP9YKLMswf6WTaza5JFScWVsohzUYqtFpwPd+62hB4x8g XyojSJWdd5oDVs23ySz2DTb4Bn9BuMGvT2uY0fbgF9W70la7zzgSEaIDVQBhi5gl9Po1 LTXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from:ironport-sdr :ironport-sdr; bh=cSZcVe6pZt3Z+p1XyU5Yi1kx7aSEESVhmG+uMlg/uyY=; b=igchXQfdZyz01rOxbkM3cnN1zw8b1LFTdaPLVY/8rm/wWH6A/mtHsU2I7hzEZbzb2J o0CEsHbLqmPpIRFJge9o2JZnb92rGH77XNeuxlbvEGzEKTS6kbO2CtqR4B2BTpD/dImL 9VNXtDnu9XKijOcgoinYndBA0U+Aw9PYwSMWeFkHsUAybQyRHQDIjk55vWAtkcWKfs2N N/776xh54IYliaQhwRUKlMxWLKXjP0hW+dZDXEnCu0xCgHH25e7yzj4nPLnGm7sgfCfy P3nTZ5J0VvenmHtp3/zvqR3Q2U1Pcac624DwBJM0bQSvgVf9SHsRBrHtUAJecNpJar2b QWMg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.180 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Return-Path: Received: from esa3.mentor.iphmx.com (esa3.mentor.iphmx.com. [68.232.137.180]) by gmr-mx.google.com with ESMTPS id v7si292239edj.5.2020.11.26.03.19.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Nov 2020 03:19:48 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.180 as permitted sender) client-ip=68.232.137.180; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.180 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com IronPort-SDR: sNM/W1s/6i06kfY+ez8q95mhLwcTDIo9OeUycpJqA+RB1bgq0E1QnY6jgGIn9NeLmbJ1LxA/Ho d7kLFuT1DGO0ycEZ84WOomkKhL7tRgpa9LCbcQ13+TyETqRQ5SS4H46m5cRghl6qbK6H2XzSQM /y7fwT+RlnpLfbqIHHfJG8IeZXv/WLIaqzV7IRDL0b1o1QNuSghCwj121WMmMganqeZKbOitIJ InEK34ok23vzWpm7CCaAPO36SWw/o3O+pRf5Bs6ci1VCJdC57cKm2nsZEEIs3E9PXEhy9TjZir orQ= X-IronPort-AV: E=Sophos;i="5.78,371,1599552000"; d="scan'208";a="55500394" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa3.mentor.iphmx.com with ESMTP; 26 Nov 2020 03:19:46 -0800 IronPort-SDR: +F7ATVPmS5MbM6x9lJCu17/X+Aj/w+bPIYzzVq3mJEM3MQLHglbvaSo+pNdswRcukrbBUrQjgZ gDMk2qioOAP/3FAi5eD2zXHWl55GZSdegujUCKduxsnB5c/qgvSjgllpqZb9sogHq3l/WPx/gU +tunybTT5aDNP1EpJf6mo5X/5popDno+JiDt0eVidbLMAjc3yKFN4EJ6oo5OHAc0IvDGJ3cV3x T8kWlSYO/acjIqgZjDmJ0xhHYErcorngP2NCWY8jAaFpVfVixD2knW/PXm7uILyRO6tqo/Ruo+ 56Y= From: Vijai Kumar K To: , CC: Vijai Kumar K Subject: [PATCH v9 00/14] Debsrc caching Date: Thu, 26 Nov 2020 16:48:58 +0530 Message-ID: <20201126111912.3037-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain Return-Path: Vijaikumar_Kanagarajan@mentor.com X-ClientProxiedBy: SVR-ORW-MBX-09.mgc.mentorg.com (147.34.90.209) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-TUID: eyJm+XmaxdFz Changes in v9: - Address review comments from Baurzhan - Rebased on latest next.(no conflicts) Changes in v8: - Fix documentation as per Jan comments. Changes in v7: - Document details in user manual. Changes in v6: - Rebase against latest next - Address review comments from Henning Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc6.3 Changes in v5: The major changes in this series are as below, - (P2) Handle cases where HOST_DISTRO!=DISTRO. This avoids mixing of debs from different distro which inturn helps the src caching logic. - (P5) Avoid downloading package from other builds. - (P8) Add a reprepro based sanity test to check if the repo contains the sources for all the debs. - (P9) Introduce a new variable BASE_REPO_FEATURES which provides means to enable or disable various base-apt features. (cache-deb-src for now) - Some fixes in (P12 & P13) Also, addressed some review comments from Henning. Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc5 Changes in v4: - Use = format instead of just using to download the right version of source package. Changes in v3: - Take care of non-existent downloads/deb-src directory. Changes in v2: - Introduced additional patch to cache deb src - Rebased on top of henning/staging4 tree Vijai Kumar K (14): rootfs: Make rootfs finalize a separate task deb-dl-dir: Cache host distro debs separately meta: cache deb srcs as part of postprocessing deb-dl-dir: Make debsrc_download faster deb-dl-dir: Download files only belonging to the current image deb-dl-dir: Factor out the mounting part deb-dl-dir: Fix skipping of removed files repository: Add a sanity test to check missing sources base-apt: Introduce BASE_REPO_FEATURES repository: Fix failures due to missing section scripts/ci_build.sh: Enable deb-src caching rootfs: Fix possible overwrite of existing resolv.conf rootfs: Handle failures when postprocess is rerun doc/user_manual.md: Document details about deb-src caching doc/user_manual.md | 30 +++++++++ meta-isar/conf/local.conf.sample | 4 ++ meta/classes/deb-dl-dir.bbclass | 61 +++++++++++++++++- meta/classes/dpkg-gbp.bbclass | 8 ++- meta/classes/dpkg.bbclass | 8 ++- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 43 ++++++++++++- meta/classes/repository.bbclass | 13 +++- meta/classes/rootfs.bbclass | 63 +++++++------------ .../isar-bootstrap/isar-bootstrap.inc | 12 +++- meta/recipes-devtools/base-apt/base-apt.bb | 3 + scripts/ci_build.sh | 1 + 12 files changed, 196 insertions(+), 54 deletions(-) -- 2.17.1