From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6750653680817537024 X-Received: by 2002:a7b:c444:: with SMTP id l4mr1557145wmi.49.1571862512299; Wed, 23 Oct 2019 13:28:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6000:c4:: with SMTP id q4ls2121970wrx.3.gmail; Wed, 23 Oct 2019 13:28:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqyUonfuQPfdVx589fLrsZXAW4Nuee3iyMoTFzsTElZI4ocLQyd7Q8EIj88g14q9tS1Dockx X-Received: by 2002:adf:dbce:: with SMTP id e14mr516717wrj.49.1571862511482; Wed, 23 Oct 2019 13:28:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1571862511; cv=none; d=google.com; s=arc-20160816; b=y448cIvHMLnkUkmOaBxVY6qF82li4Sy1qLtYg+ty3MMMZARSNQ+OpFT2PwXjMteOSr UH5NKUKwRi9DSUi4lAQY3ZBlugwzybx07i5g9+SthSO15FE+skQ+78Cbpdar0SCLFxt0 uvCg/ZxeWB/MQZfY1ljahzBSgPn9O32da6g6rnQRHFhZ7BKdicLILal6+C/Lc7Yps0di UjGWqglZqzuC7ZYVrCGJFD4DyKIm5FEMbDixisTP/e4lSlb0LwDiF7WGd905P2+rNKTi d6JpqmMrOhX/t9CYgMGvJFciL2II4otYHpq/LrlvabgUhE9f0Ty1XP6vktWgkGmw+eU6 WKgw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:from:date; bh=ezhuw0A1W9IiJ6uNBLVI4cFFr2HkqXqPWC951T8hans=; b=K6pcYBlEPP6BwaYzb1fCSqEzJ+r2qmJTeEhzslsc6UPSqeCgUj+CmSuCgEyt959vTU 1tplSKAGgTa03Z9687F7dBhAcCYBjMEhGhegBQ79W/A4flo3gzTSpommfaxZer7jAhZc JTwf2D8uGP7Lkx3j0UEcpl4yB8zzUWepMrUljPWQbz45Au+VLWpc74QMgsGEG+2kuDNK fOaO+Mds1q84m+HuLJU2csCHVxEyONL7z6dTiIJ3mcvmsP0+EeJ2tDAWNiWHzc+UkR/C uUyF5BXWJ/JeuFtx/6sR/5QCEJ5betPuSHeHqdJTN7bQFOiSQfJu5TStxc1Tt3+sEs8F wfyA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id e17si1004871wre.3.2019.10.23.13.28.31 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 23 Oct 2019 13:28:31 -0700 (PDT) Received-SPF: neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 85.214.156.166 is neither permitted nor denied by best guess record for domain of ibr@radix50.net) smtp.mailfrom=ibr@radix50.net Received: from yssyq.m.ilbers.de (dslb-084-061-174-150.084.061.pools.vodafone-ip.de [84.61.174.150]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x9NKSTil013083 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 23 Oct 2019 22:28:30 +0200 Date: Wed, 23 Oct 2019 22:28:24 +0200 From: Baurzhan Ismagulov To: isar-users@googlegroups.com Subject: Re: base-apt work in progress, RFC Message-ID: <20191023202823.gn2vqsaokj2lfzmz@yssyq.m.ilbers.de> Mail-Followup-To: isar-users@googlegroups.com References: <20191022174359.40fb4045@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191022174359.40fb4045@md1za8fc.ad001.siemens.net> User-Agent: NeoMutt/20180716 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: zgWGXy7pQMuv On Tue, Oct 22, 2019 at 05:43:59PM +0200, Henning Schild wrote: > i now have a working version that changed a few things around the > implementation of base-apt, to make sure it will also work when some > rootfs clear their package cache etc. Ok, good to know. > So far base-apt mixes together packages from several rootfss. If these > rootfss had different versions of a package the latest version of that > package would win. See meta/classes/image-cache-extension.bbclass, we > are mixing the image with the two buildchroots. > The preferences and sources in these three rootfs can lead to such > mixing. When this base-apt gets used for a rebuild such newer packages > can leak to rootfs where they were not in the first run. Do I understand correctly, you'd like to cover e.g. the use case below? * multiconfig card1-arm64-buster uses buster with pkg1 1.0-10. * multiconfig card2-armhf-buster uses buster with an older pkg1 1.0-9. This is a relatively complex policy issue. I'd suggest to start with the simplest case first, then add features on top of that. > I see two general directions to solve that: > 1. align the sources and preferences across the 4-tuple image, > buildchroots, sdk ... and have a base-apt for every image-4-tuple Could you please give an example of two such tuples? I don't see how that differs from the image-MACHINE-DISTRO-ARCH case. > 2. have a base-apt for every rootfs-PN-PV For the example above, would that mean the following? base-apt/debian-buster-pkg1-1.0-10/... base-apt/debian-buster-pkg1-1.0-9/... > 3. if reprepro has deduplication accross distros/suites possibly have > all "base-apts" in one repo, and make every instance a "suite" > so instead of "buster" think > "isar-image-base-MACHINE-DISTRO-DISTRO_ARCH_1.0" > Now if they in the end are all buster and use the same file, we only > need to store it once and can still store differences. Same for > sources, where this even works cross-arch. > > I think 1 is not a good idea, isar currently allows sources and > preferences to differ and it is probably a wanted feature. I'd suggest to start cache_base_repo without preferences and add the feature later. And when we add it, keep the cache_base_repo tooling separate from the bootstrap tooling. I think the users would like to use a certain codename (e.g., buster) with some explicit pinning (e.g., pkg1 1.0-9). That could be described statically in e.g. a json file, rather than resolving dynamically via preferences. After you've cache_base_repo-ed that, apt would get the version in base-apt, making preferences unnecessary. > 2 is "clear" but we have a lot of those repos, which might be hard to > handle, if we think people pull updates from there I'm not in favor of that, either. > 3 is currently me preferred version if deduplication is in place, i > think even without it Deduplication is possible with the Debian repository format. reprepro supports it. I think the first step should be exactly how Debian does it, i.e., BASE_DISTRO/dists/CODENAME, with different DISTRO_ARCH packages in BASE_DISTRO/pool/*/*/*.deb, without preferences. > I am not sure whether the solution to that problem should be part of > the first patch-series. It is not a new problem ... > In fact i will probably sent my current state without the "deep tree > download", which would also be something on top. If it is not worse than the current implementation (which I hope is the case) and passes the CI, I'd like to see it. With kind regards, Baurzhan.