From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6878157486786871296 X-Received: by 2002:a25:d34a:: with SMTP id e71mr3654619ybf.229.1606291407390; Wed, 25 Nov 2020 00:03:27 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a25:3806:: with SMTP id f6ls809661yba.4.gmail; Wed, 25 Nov 2020 00:03:26 -0800 (PST) X-Google-Smtp-Source: ABdhPJyAeDR48C9tUu5CBWc0ZqjU72m3VQWWmHMfO3vnSQglBR5SuOziS0mxMs8O2RSO/zEUz5s6 X-Received: by 2002:a25:d9cf:: with SMTP id q198mr2171847ybg.243.1606291406749; Wed, 25 Nov 2020 00:03:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606291406; cv=none; d=google.com; s=arc-20160816; b=eOIDDxEk4fNXfXGhTRqziRcKzNKU6qYbPcabLx30rBdNHzx+sJwct77MUhQsw5DZ9R da6ESOWXxKyV0RpFtFnbOfcHlMKJ4i399Ri496j6YxOXqkWxB1rj2vkwwkI2AYOH0+Ko JW1NlkAb4r1LSqn8+OE9/pZrlcCJKgycWvAGKq0+bmQDNYMrvoL/jbUCVqucSu4idX/j i+wK9rKvXvRJZgRrEVvsXEn1karJljECsmy0WngNaAjHlkzpvnS8mn3OHxM86gYn1z9d NcFApXbPDKc3Dd36agkf2E/93XzvDrEwpP59rdjLpU8gUpl1io0133diGhWZ+07E+9Pc u+JQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from:ironport-sdr :ironport-sdr; bh=04nEdD9GZYSymStf2woAo9K2m2lTDiWk9f+Wu7ndHlU=; b=Mhv6U1IU8C4viJDLOoCsgdZkszf0dYVvo1K8QlU3AdF4tg0Np1a/LLIf/MjRmovu0u 0c0Fcfaw7oERM4cKKpPV2PdL9JDAsf3mAvHpxw/9cY/txzjjlyFcqxN+Hb/pnnEhlppl 3kU0Fh+kjjjdebUpwsZ5AMURDLP/JofGn9gBxC17dcic3Hbl/y8lO+0NwDGIB2unQzks cc9z33aYqN0A7nyeYr5e0++8cV/jitMWCx65I0XvRV2UOxeodtyZGlr4hgL0ZnfkatDY gk+lupYpE0n89l6f6KXZIM8wq5Z2OUVo/zfD6SmiwJhnMjvB6+94pyZ8sWaF4nQu7aXy jbPA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Return-Path: Received: from esa1.mentor.iphmx.com (esa1.mentor.iphmx.com. [68.232.129.153]) by gmr-mx.google.com with ESMTPS id 199si77218ybd.5.2020.11.25.00.03.26 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Nov 2020 00:03:26 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) client-ip=68.232.129.153; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.129.153 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com IronPort-SDR: hmdrx7kEhdOgx5tOBE6prfjG2i2w85+j7VXCmAysGKMAs295U453QzrPcTi4gn41vSvc84o4A8 /g0mUwpTp6kVuTzIIlLRPUT0EPabrw/KDoCpfD8NrtXfYb44id4jOV4AMKU64KAbKlyQmRpVvY qrAlVYhOeCTtEv2hrLUYIBvJDIDxdGJtnqq64nK6+f5zrhfRe+gdkxbKE3AkLTKrFTKLMIhFG2 kiFSEbGjTfz/O/pPc4rJL6LpqqmEuzR9/fRfrKzqXGFxsn3mc+wdlcRVGD3txPdbT/rCZzUVht Pxw= X-IronPort-AV: E=Sophos;i="5.78,368,1599552000"; d="scan'208";a="57762463" Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa1.mentor.iphmx.com with ESMTP; 25 Nov 2020 00:03:25 -0800 IronPort-SDR: lPSNrkNj3jwhbv79l5D3v6FL3cMFXtLJP33XHDFZlc2lxyCZFjG6g8Pgl+cqNbgC4juIGMnWsk hK0suH9GEvnyX6lMDeV9eTyAYNRI6WhtKq3JljYX8Ezn7wixP8EVsnjhP4NrtrGGI879s6MgqX KYqduXmKUUZy3OahcMqI3DqXbtWpc8tEXT2VkCHZ08JLEhmiOsR3wzHyUUWGCVS3pAHk4NJ17j w9GxpF6k0SWtniqzX176tc6lbqV9bWOOkBG7Bf81IgaMJWBOvzf1LDW6ZHHg5ZyTbgGp1w26w2 H9M= From: Vijai Kumar K To: CC: Vijai Kumar K Subject: [PATCH v7 00/14] Deb-src caching Date: Wed, 25 Nov 2020 13:32:47 +0530 Message-ID: <20201125080301.15818-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain Return-Path: Vijaikumar_Kanagarajan@mentor.com X-ClientProxiedBy: svr-orw-mbx-04.mgc.mentorg.com (147.34.90.204) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-TUID: FMOpj4w091kB Changes in v7: - Document details in user manual. Changes in v6: - Rebase against latest next - Address review comments from Henning Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc6.3 Changes in v5: The major changes in this series are as below, - (P2) Handle cases where HOST_DISTRO!=DISTRO. This avoids mixing of debs from different distro which inturn helps the src caching logic. - (P5) Avoid downloading package from other builds. - (P8) Add a reprepro based sanity test to check if the repo contains the sources for all the debs. - (P9) Introduce a new variable BASE_REPO_FEATURES which provides means to enable or disable various base-apt features. (cache-deb-src for now) - Some fixes in (P12 & P13) Also, addressed some review comments from Henning. Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc5 Changes in v4: - Use = format instead of just using to download the right version of source package. Changes in v3: - Take care of non-existent downloads/deb-src directory. Changes in v2: - Introduced additional patch to cache deb src - Rebased on top of henning/staging4 tree Vijai Kumar K (14): rootfs: Make rootfs finalize a separate task deb-dl-dir: Cache host distro debs separately meta: cache deb srcs as part of postprocessing deb-dl-dir: Make debsrc_download faster deb-dl-dir: Download files only belonging to the current image deb-dl-dir: Factor out the mounting part deb-dl-dir: Fix skipping of removed files repository: Add a sanity test to check missing sources base-apt: Introduce BASE_REPO_FEATURES repository: Fix failures due to missing section scripts/ci_build.sh: Enable deb-src caching rootfs: Fix possible overwrite of existing resolv.conf rootfs: Handle failures when postprocess is rerun doc/user_manual.md: Document details about deb-src caching doc/user_manual.md | 25 ++++++++ meta-isar/conf/local.conf.sample | 4 ++ meta/classes/deb-dl-dir.bbclass | 61 +++++++++++++++++- meta/classes/dpkg-gbp.bbclass | 8 ++- meta/classes/dpkg.bbclass | 8 ++- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 43 ++++++++++++- meta/classes/repository.bbclass | 13 +++- meta/classes/rootfs.bbclass | 62 +++++++------------ .../isar-bootstrap/isar-bootstrap.inc | 12 +++- meta/recipes-devtools/base-apt/base-apt.bb | 3 + scripts/ci_build.sh | 1 + 12 files changed, 191 insertions(+), 53 deletions(-) -- 2.17.1