From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519381492065370112 X-Received: by 10.46.17.72 with SMTP id f69mr156390lje.41.1517925824448; Tue, 06 Feb 2018 06:03:44 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.46.12 with SMTP id u12ls1015239lju.4.gmail; Tue, 06 Feb 2018 06:03:43 -0800 (PST) X-Google-Smtp-Source: AH8x226beI7T0nIyrdsSCdfvTp88Ob5CJ7JLZxfriZd/Xhyjp2qrDlEMETDeW5XSLBnANPdj7XI/ X-Received: by 10.46.34.69 with SMTP id i66mr151481lji.10.1517925823841; Tue, 06 Feb 2018 06:03:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517925823; cv=none; d=google.com; s=arc-20160816; b=jrMlqvGy2vjCXGoIgUsv9oS7CumeEOPdMCUY7HScnjlPYGuVDyOVVWX/aWgMPIhCEH ErniSlGHNVzVml1nW8P5Jn1g1lr0tt/DytuM8p0cibl6JXWDoXKqgTyDeF7LbpiZm5W2 4amn+xMqol14z3hlW5D7ertbudH2ZL5FM63DpjLNCpyRVF/QfgP0M01qsmKzHqyYYT4H zluzy6yNi48jhhaK+2md7IhLiMFdSNY73lt1U6qLAPDNRS+TLjU+KocBl2qw8ziQtIWq zQ68mWcOABMMso38F5lA/3Neqydts8CsjYCENS9PWsi4/j01wzzZkBw/ZAXElWKmkYcS vUzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :dkim-signature:arc-authentication-results; bh=IsIzscAK0/X6v8Vt001wYrMuQbZm726W+J2UjIIG1Pc=; b=kr4R9mLSP0DddL1bPm/yzjpDqgRY4OP67GxGh7cMksZl3I6HAJV+bcZK6rAjKvDrHD BOGh+nNwRW0ZDVwrjU4RDFRMl0OPFg1tKfU6+Mb4JsZPYSqCU7940rQjhAcdLTQechk7 TA+hRKbdW0YCBxLrT0XqofGRBUYodHOvE8OQey86rQuDr6391M2aX9RChnJt94QwNLj8 DzTji0m+qM+1o70gU0NSvEkVBr3LfmAmO4m51I/os86KarxDI+RgIoaSb40BgBUxjZsk zXvMYTDsxxtfnoSwQiorAzQneSwwodyQJbqCI4hniP7NSYrHGYhWs6/gRhqkk4K5mbLc mkuA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=TOV2YVvq; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::243 as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com. [2a00:1450:400c:c09::243]) by gmr-mx.google.com with ESMTPS id o26si94500ljc.5.2018.02.06.06.03.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 06:03:43 -0800 (PST) Received-SPF: pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::243 as permitted sender) client-ip=2a00:1450:400c:c09::243; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=TOV2YVvq; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c09::243 as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: by mail-wm0-x243.google.com with SMTP id f3so4075780wmc.1 for ; Tue, 06 Feb 2018 06:03:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding:content-language; bh=IsIzscAK0/X6v8Vt001wYrMuQbZm726W+J2UjIIG1Pc=; b=TOV2YVvqdeMbgWpBms7eMGumdY/ZxmY0dUzbit0yETOq06YXrrWoaW4lN+2543knMs TFsuIx9AvnfEHeOmgLEKXMG0aPwBtyeq46T3bQnX3c0SQE37s04fM6lcrkvhkUSdP3UO 0/r83CllUotxLr6oG8Xd3wsJeI23fFE5RRsfESl+EmSISLCkOhFs94ERwgihKJxF6oTB UxsAsfJMfDbzi1GkiVGPmGm6DjjRGxo/Ys+DRFHNm7e8HIWwUDBzGpHtvw1iJGRSUR8d XQ0kwgALBv3UGSzTrSffH5mjmXpSVK8TapvuO7kiaUG1HUFRZEwmlugIpSx98G5+lQYX QJtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=IsIzscAK0/X6v8Vt001wYrMuQbZm726W+J2UjIIG1Pc=; b=aO+MbmZiQsfwhQ7f/nkmlP22edIn/x9VtDHClm4yvpAKFljxHoH/8L12x2Lq7QVd29 clAkMFsUaStKlvAHm2SkSOOp4t1BlMdi2lqOmvZsukvoF7WqAFJoxjGIsRYLPqiYFQhM r7eHcEzu78OVlep9MPkjuF5nNjLIiipvWodZwcHkMUV8fBPBXLoo+yzLPMwwC3VSgAGI 0OA8hSqOII1VC2VGn2P2Vz/uv6aO1ZdSRlM924EvaY6tq9Bb27v+acqn0hcw3n/mm+fv FZ6VEyNFGwfA1PBSXvkXo46iHc44pT63QMzPS/Kmpmo7Z4AZtFE4l2aT1fPkVxPh1dBv poPw== X-Gm-Message-State: APf1xPAqC5rc4aB0iALSYAXPpUtGxBjZ4yIbUOTe9tVyFfo5W4Cxl1EH dv8Kprhlr2GQ1+qfcSZ/anuLYIQI X-Received: by 10.80.212.154 with SMTP id s26mr3787382edi.268.1517925823384; Tue, 06 Feb 2018 06:03:43 -0800 (PST) Return-Path: Received: from [192.168.43.209] (ip-109-41-194-93.web.vodafone.de. [109.41.194.93]) by smtp.gmail.com with ESMTPSA id 60sm9141939edy.36.2018.02.06.06.03.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 06:03:42 -0800 (PST) Subject: Re: [PATCH 0/6] Local apt cache with aptly To: Jan Kiszka , isar-users@googlegroups.com Cc: Benedikt Niedermayr , Cedric Hombourger , Alexander Smirnov References: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> <3c6785b1-1b80-e406-e499-515f89b055b9@siemens.com> <4ec03380-fb4f-ae64-0cde-4172235f4e7b@siemens.com> From: Benedikt Niedermayr Message-ID: Date: Tue, 6 Feb 2018 15:03:41 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <4ec03380-fb4f-ae64-0cde-4172235f4e7b@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-TUID: ZOaJaoJVN8I/ Am 06.02.2018 um 14:28 schrieb Jan Kiszka: > On 2018-02-06 13:39, [ext] Jan Kiszka wrote: >> On 2018-02-06 11:09, 'Benedikt Niedermayr' via isar-users wrote: >>> The mirroring solution does the following: >>> - Aptly generates a mirrors for upstream repositories as well as snapshots and also a local repository for isar packages. >>> - Get a list of all packages for mirroring before rootfs and buildchroot is beeing created. >>> This is done by running an own bitbake cooker for parsing the recipes by taking append files, >>> configs, datastore and also selected layers(bblayers) into account. >>> This solves the problem when upstream repos getting updates between rootfs and buildchroot creation. >> This will not address the case of generated debian/control files and >> their Build-Depends, right? I'm not a fan of trying to extract the >> information in such a collection run, simply because of such tricky >> restrictions. Therefore, my proposal is to postpone any up-front >> fetching to later, if at all. > Hmm, does your patch address Build-Depends at all? Can you explain how > they make it into the aptly mirror filter? > > Jan > > This will not address the case of generated debian/control files and > their Build-Depends, right? I'm not a fan of trying to extract the > information in such a collection run, simply because of such tricky > restrictions. Therefore, my proposal is to postpone any up-front > fetching to later, if at all. Yes you are right! That in fact is a problem which still persists, and I will try solve it soon. But I have also not tested it. > Well, how does this series relate to the discussions Alex and I had and > the roadmap I proposed? And how do you see it compared to Cedric's proposal? Since I was developing an approach of aptly, for own education purposes and PoC. This patch series is only a PoC,  for running builds against a prefetched mirror. AFAIK currently no approach is based on using aptly. This series should give everyone the knowledge that aptly is also working, and also demonstrate some extended features compared to reprepro. Cedric proposal is not using a prefetching mechanism, or do I miss something here? And Alex shared his vision, by fetching all packages within a single step/recipe. At last I think, it is not wrong to have different approaches in order to show the pros/cons more clear. > Hmm, does your patch address Build-Depends at all? Can you explain how > they make it into the aptly mirror filter? As I just mentioned... Build-Depends where not adressed yet. I've focused on a basic prefetching mechanism. Today I will have a deeper look into Build-Depends, and give you reply. Benni