From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6878157486786871296 X-Received: by 2002:a05:600c:2541:: with SMTP id e1mr4145373wma.185.1606311787431; Wed, 25 Nov 2020 05:43:07 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:c689:: with SMTP id j9ls2845619wrg.0.gmail; Wed, 25 Nov 2020 05:43:06 -0800 (PST) X-Google-Smtp-Source: ABdhPJxIiSase/rK/0wNCu1Jtte2ZQXTP2NwgxvdRQsPwryNohvIUyVcqOEx8ABT1YwCJdaL9pG4 X-Received: by 2002:adf:f110:: with SMTP id r16mr4200347wro.350.1606311786266; Wed, 25 Nov 2020 05:43:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606311786; cv=none; d=google.com; s=arc-20160816; b=rbk6/jEH9Zy5SI1OpLLm30IAssA398u6zuSYO88Z7zxoRz8zCOGN+8odaasOmtIIHn 75ooya7txATKz9cvhl+n8F5xVXPCq6y8WPQkfOKl622LFFaVJlOU1U0u5c9jYk79Hs85 X/75ioPEW9MJqJJD9daZQOJ5GUiI8+vA0lfR2JLC0qFOdirU8nPROodaLZNtOzj3rTZI tu++xdJwvLna6IuvmwOCgZ0Tabmzwnm62vX0akqZ/HSeIcRk27MOKkU2dTmCQqyyjLHi inaASE23NrvCGYABpoInGY1bej3xYoQRnhP/5qpJeQlnLnHg7qPwHW2rcNtzA14GRW9q ZpvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:message-id:date:subject:cc:to:from:ironport-sdr :ironport-sdr; bh=wZeW3CJvQhL6fc5T/AfCCnWzfNLH9XO4C8SYLdgaOJg=; b=qImlVDSJq02xaOS9kPrl+IYXPVUdPv/G3onUXwHygOIa2+PBSEUPKhhnJmjdad0g5j Blni18CgiEPVhqaCVa4Y1QBm98u2CeUBqIVwQhnlIBbhu2IUBqR2EhrdavvL8qd+TYws 9KFXBG5cbRjY1uyAxG1XK/lw4Sr85OGpbmqpkSNyeNOCy9BWnLqHFx6cO8uE9uGndP0u GzNte0lG9UFBjNi1ANwQV6pNgHxXHslbB8nfLCEFrPKniXWvfm/HreFE93nPYBcCSOHt BQKdz78NDuAQSTJ5kAaL72rlrTlxD0rRNb8l70XAugGqQ/46Bbf8LxLpwkV5hgVPw7HQ x6nw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com Return-Path: Received: from esa4.mentor.iphmx.com (esa4.mentor.iphmx.com. [68.232.137.252]) by gmr-mx.google.com with ESMTPS id c20si60917wmd.2.2020.11.25.05.43.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 25 Nov 2020 05:43:06 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) client-ip=68.232.137.252; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of vijaikumar_kanagarajan@mentor.com designates 68.232.137.252 as permitted sender) smtp.mailfrom=Vijaikumar_Kanagarajan@mentor.com IronPort-SDR: NuhmvQ9HxikZAgBbdBwRNGEL4PVa08ttDuVJ/KbdIO8yyLyyVKOtO+bJVDAi+pjQLgd3fbKbRG /eJIMsovdMyvY4v2V7s1LX56dgKF4Os4qj3CzrnoFzMAHUiye0Sx7Glg7nLXLBlGuWyDR2bVNl xWDp6beK9orcC3o7AuLWFDuJmKPzuA1/7KTUwYW5B6wKgrkhMLS7WNSgnWQUteT9y4UeAcliGB cO5lnKoWhVtYasTMw97ieGrM6YckqMm36pJJBbdotoZVoJo+XRAs+mY6YjeXMfabKXDMvMWmUl ewE= X-IronPort-AV: E=Sophos;i="5.78,368,1599552000"; d="scan'208";a="55597743" Received: from orw-gwy-01-in.mentorg.com ([192.94.38.165]) by esa4.mentor.iphmx.com with ESMTP; 25 Nov 2020 05:43:04 -0800 IronPort-SDR: inId7VggM63sUQMrSJjQYA6+Hje/hQmVRpugWZPlPj7TEl/vGxKS3xkaHj0Idk380Zn9CAKAeY HxBGNiA0tZQ9EdP6FouYNjRLWDCUR6Zm6bzv/9Vjjsl+DVCdbXbLHqZrrI8kH5aG4BblFuTBmE QIlRFxTHlTY/ZGkP4RfYzvBnVi4mrCz2ob1J3q8GvaJQvHSVHzIOp3KpqFMUbG2gGDXvUUgRSR bWEsHXMhFcMcz03PseURmJcKNyEEuKgzyLI24yKNBN8ymPK0y9P18GYys/3dR+qS8e2Gr9FTXF Bek= From: Vijai Kumar K To: CC: Vijai Kumar K Subject: [PATCH v8 00/14] Deb-src caching Date: Wed, 25 Nov 2020 19:12:22 +0530 Message-ID: <20201125134236.25878-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain Return-Path: Vijaikumar_Kanagarajan@mentor.com X-ClientProxiedBy: SVR-ORW-MBX-06.mgc.mentorg.com (147.34.90.206) To svr-orw-mbx-01.mgc.mentorg.com (147.34.90.201) X-TUID: ZQPZ7RnDG02H Changes in v8: - Fix documentation as per Jan comments. Changes in v7: - Document details in user manual. Changes in v6: - Rebase against latest next - Address review comments from Henning Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc6.3 Changes in v5: The major changes in this series are as below, - (P2) Handle cases where HOST_DISTRO!=DISTRO. This avoids mixing of debs from different distro which inturn helps the src caching logic. - (P5) Avoid downloading package from other builds. - (P8) Add a reprepro based sanity test to check if the repo contains the sources for all the debs. - (P9) Introduce a new variable BASE_REPO_FEATURES which provides means to enable or disable various base-apt features. (cache-deb-src for now) - Some fixes in (P12 & P13) Also, addressed some review comments from Henning. Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc5 Changes in v4: - Use = format instead of just using to download the right version of source package. Changes in v3: - Take care of non-existent downloads/deb-src directory. Changes in v2: - Introduced additional patch to cache deb src - Rebased on top of henning/staging4 tree Vijai Kumar K (14): rootfs: Make rootfs finalize a separate task deb-dl-dir: Cache host distro debs separately meta: cache deb srcs as part of postprocessing deb-dl-dir: Make debsrc_download faster deb-dl-dir: Download files only belonging to the current image deb-dl-dir: Factor out the mounting part deb-dl-dir: Fix skipping of removed files repository: Add a sanity test to check missing sources base-apt: Introduce BASE_REPO_FEATURES repository: Fix failures due to missing section scripts/ci_build.sh: Enable deb-src caching rootfs: Fix possible overwrite of existing resolv.conf rootfs: Handle failures when postprocess is rerun doc/user_manual.md: Document details about deb-src caching doc/user_manual.md | 30 +++++++++ meta-isar/conf/local.conf.sample | 4 ++ meta/classes/deb-dl-dir.bbclass | 61 +++++++++++++++++- meta/classes/dpkg-gbp.bbclass | 8 ++- meta/classes/dpkg.bbclass | 8 ++- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 43 ++++++++++++- meta/classes/repository.bbclass | 13 +++- meta/classes/rootfs.bbclass | 62 +++++++------------ .../isar-bootstrap/isar-bootstrap.inc | 12 +++- meta/recipes-devtools/base-apt/base-apt.bb | 3 + scripts/ci_build.sh | 1 + 12 files changed, 196 insertions(+), 53 deletions(-) -- 2.17.1