From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6816610675337461760 X-Received: by 2002:ac2:48b1:: with SMTP id u17mr1424900lfg.187.1587115851509; Fri, 17 Apr 2020 02:30:51 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:551d:: with SMTP id j29ls242659lfk.1.gmail; Fri, 17 Apr 2020 02:30:50 -0700 (PDT) X-Google-Smtp-Source: APiQypKGgzzigI/1umbClpgbb9WOP6uysWkX/vagfSzjMz46IkBUVYoUy/Trg3E34QtqUHbhYkjk X-Received: by 2002:a19:d84:: with SMTP id 126mr1471558lfn.35.1587115850543; Fri, 17 Apr 2020 02:30:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587115850; cv=none; d=google.com; s=arc-20160816; b=iXgWUnPRYysLc3sfWo+u12JVvNJcBLMoTtUeXt4fpYAz9+Ti0zOmO/rX/qgAPLn4z6 hOSm5ePhLy92HfFnw/5x8Mtk7vyg6y+ic/ExIL+AB8zXGOXZ6xa7lb63HxeEH0cxRI0K +jT0T28RPr4Rc2gFKzL8t927Knki7rWlFhRQ0RKMeeRShvt949g6QUGQGz4iwQGAStB9 qL20Vox8PjOFIDD/Ce8ibw1jeTUlruFpBoh18nIa3RN7ESkVa75lSxqr9r85hhnUeafM hdDHtmtsYzxiol9o7k0NE4cgABuYtKZaqHubQp2RtTlWC5UkfCCc/SYIn4LV2InwbyRs YKCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature; bh=9X4/4UQH3g3y0/GPC06QuqXbWsUjBquENnpCvAiWjrI=; b=dSTNAaPBxXjEMrSJfFwxvpdK3uOEhfxwZ1PyB+QYL52o4KXHF7YN8CKfx3m6HPOLOJ 8ieiCpJjxUGSaifgqaQXR+pCK5NIMJisHzj+lRMENZeoE52gXvZ/0+wOcEx/bl0RmWhQ u76iSkeItFdjrNK7C++gDWN0jL2yDKfEeGya0NCy8skfGZoqDq3CDrsZBuhw2YGC5hqs UsOBu04TH0nG+c3rup9Cny9Wf4fry6vNrC5phOTCQzAvMRn0aOovsj8zXP8R3N4hEsP6 aJSmL/C6RxwEY7lazS4up/i1zxtZE9CX4X6EKGMSt+bN1Qi1fm2y86nV4SHJxy4lC4JE U+ow== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=WF53zfI4; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com. [2a00:1450:4864:20::42a]) by gmr-mx.google.com with ESMTPS id p5si1271478ljj.3.2020.04.17.02.30.50 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 17 Apr 2020 02:30:50 -0700 (PDT) Received-SPF: pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) client-ip=2a00:1450:4864:20::42a; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=WF53zfI4; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2a00:1450:4864:20::42a as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-wr1-x42a.google.com with SMTP id x18so2287908wrq.2 for ; Fri, 17 Apr 2020 02:30:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=9X4/4UQH3g3y0/GPC06QuqXbWsUjBquENnpCvAiWjrI=; b=WF53zfI4iU8mOql1Uv+yRf2YvvxmtlvmpPBT0susY0xC8lACFsXkPU6KGgEoGV9MhS amFPlVVEvor5kIewNFGpGtjo2okLK4JKp7f/KjPC2f6HTOg6TWoK7hl4+S/3RLe6YXgE oszJdBn4NjG6kavVB26emMxrSwp+1WdHZPgb4/6olZZH2rIsv8qD/lCWotwqb7mZrCqo v0ZLO1YaSGOTk0xVKOFx9jAdt3wvWxjo/5QRLUXcmA3HyN9oHFq06xS+78LTnVCJYqxS xCDJfug4uThW2qpLms9HJH0E6pzP6TAKJ0coVB3wEQ/eSaTwxIWyJrKgQmDCAdhlGW3z KMKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=9X4/4UQH3g3y0/GPC06QuqXbWsUjBquENnpCvAiWjrI=; b=q4DuP5wZWmS1LNzgXA9ViMmNm8041XEC0H/ESkoY1c3pwtpj9jc3pv9nXGzrMV/N3F 4+zOdHia3mxGW1e8RF4Vt0DOqNgetV4+Q9OEduLiGRmJAJgFOlnEUPQT+Ob/UNBVIJxr AlAIIV7LqhTg3bU/YqYVX5QZkc79uyvAxntABlxDvg/9SwLC8GKhPl6l9NtGSCZzJJHR LhjRCNwkvIWUkgYo61wIm1zwUgkweRvd8f/AFReAyjL1hLGVXnNN8vWo5FKzn4lxsdjO 4DYtamtX6N4nCotZOG94QbvyJWGSvZVEJngfKka/JWvOT9b8kwuyNrQSOk1Dgf8ry/5W atzw== X-Gm-Message-State: AGi0PuY4y7+Nv+f0DxparP4G8wmUePLx3lFUmZP08di+4eEoP1YQn+uh fXwMABsEbIW7C3suk+p5ZTTD2IiNG1w= X-Received: by 2002:a5d:4e02:: with SMTP id p2mr1807226wrt.302.1587115849602; Fri, 17 Apr 2020 02:30:49 -0700 (PDT) Return-Path: Received: from oxygen.mgc.mentorg.com (nat-sch.mentorg.com. [139.181.36.34]) by smtp.gmail.com with ESMTPSA id l15sm6836256wmi.48.2020.04.17.02.30.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 17 Apr 2020 02:30:49 -0700 (PDT) From: Vijai Kumar K X-Google-Original-From: Vijai Kumar K To: isar-users@googlegroups.com Cc: henning.schild@siemens.com, ibr@radix50.net, jan.kiszka@siemens.com, Vijai Kumar K Subject: [PATCH v5 00/13] Deb-src caching implementation Date: Fri, 17 Apr 2020 15:00:27 +0530 Message-Id: <20200417093040.15130-1-Vijaikumar_Kanagarajan@mentor.com> X-Mailer: git-send-email 2.17.1 X-TUID: /ejNhnf1QHG+ Changes in v5: The major changes in this series are as below, - (P2) Handle cases where HOST_DISTRO!=DISTRO. This avoids mixing of debs from different distro which inturn helps the src caching logic. - (P5) Avoid downloading package from other builds. - (P8) Add a reprepro based sanity test to check if the repo contains the sources for all the debs. - (P9) Introduce a new variable BASE_REPO_FEATURES which provides means to enable or disable various base-apt features. (cache-deb-src for now) - Some fixes in (P12 & P13) Also, addressed some review comments from Henning. Git Tree: https://github.com/vj-kumar/isar/tree/vijai/debsrc5 Changes in v4: - Use = format instead of just using to download the right version of source package. Changes in v3: - Take care of non-existent downloads/deb-src directory. Changes in v2: - Introduced additional patch to cache deb src - Rebased on top of henning/staging4 tree Vijai Kumar K (13): rootfs: Make rootfs finalize a separate task deb-dl-dir: Cache host distro debs separately meta: cache deb srcs as part of postprocessing deb-dl-dir: Make debsrc_download faster deb-dl-dir: Download files only belonging to the current image deb-dl-dir: Factor out the mounting part deb-dl-dir: Fix skipping of removed files repository: Add a sanity test to check missing sources base-apt: Introduce BASE_REPO_FEATURES repository: Fix failures due to missing section scripts/ci_build.sh: Enable deb-src caching rootfs: Fix possible overwrite of existing resolv.conf rootfs: Handle failures when postprocess is rerun meta-isar/conf/local.conf.sample | 4 ++ meta/classes/deb-dl-dir.bbclass | 72 ++++++++++++++++++- meta/classes/dpkg-gbp.bbclass | 8 ++- meta/classes/dpkg.bbclass | 8 ++- meta/classes/image-tools-extension.bbclass | 4 +- meta/classes/image.bbclass | 43 ++++++++++- meta/classes/repository.bbclass | 13 +++- meta/classes/rootfs.bbclass | 62 ++++++---------- .../isar-bootstrap/isar-bootstrap.inc | 12 +++- meta/recipes-devtools/base-apt/base-apt.bb | 3 + scripts/ci_build.sh | 1 + 11 files changed, 177 insertions(+), 53 deletions(-) -- 2.17.1