From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6740510031426355200 X-Received: by 2002:adf:ce04:: with SMTP id p4mr2088308wrn.130.1569490828520; Thu, 26 Sep 2019 02:40:28 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:e304:: with SMTP id b4ls461906wrj.14.gmail; Thu, 26 Sep 2019 02:40:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqx5xNyuyyfzHi9EAWiIp1aQ2iBUGOuT96s/bGnWqp9dHcSf+yyFewxPPOkpiWiV8c1v6WAN X-Received: by 2002:adf:e9ce:: with SMTP id l14mr2269765wrn.264.1569490828092; Thu, 26 Sep 2019 02:40:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569490828; cv=none; d=google.com; s=arc-20160816; b=H49gkVDzyAi9ul5M4I8cCzmoZrWCMV/hLIhs6rRqxRWjzqXjXf25Q+5ri/ApSkMoAn YDcnsxm4GntaIX+FP7IultbVmdcF5UUJBl7wSgcxQilfvsbC7tokyhtZu3itqpqHLT2K RX8q5B4xKQiZYBZLMgTkRZbp0mQFZdNyrvKd6wrd+lw4Z7o6La8ohL93/rnsc1MH8CdD AhBKx0/e6Qhu2GlnLeiEuJ72lxfiZCqeqz75JPu9U6IZIiUx4DMD74NRSKITDbYFPu5A UvChEtnzLGrFpjpJjvEDIrXwWzEpYYz3U4JAqYnRX+2mcbm2HUmlxejV9tdkZ5tQH3rE YqnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=2zK+2ZnXNHCZTTNgnAqk5EeIc1q46Qg7vxmiTvAo9n8=; b=i2f+SCosVU9nH8D5GTdKKwsnf7yOhWVxIbw1O0k5YZFgNTjYs+n/EDg7ZbHsvcrHPc cisxdQu0FDlW1YSwzvDbL5R05Vj5/TiVr+td/HAiKBUZGVoYG7g8qBcHe+qdOfNX2mv6 JVyYD5C9sWKKEF0f+0T0RQ0clL593Y+6BCg31/L/OpWGlpy9tFBtdGEcb0k2K6unycak Vwc0FNQjF9C3j6COZURwUu3WMDwR7mvGeG1wyMqh3dQNY1yp9GcXOa2FYQrkEkAXCLRh Fi8v3gtepvK+HX5pTodvP0u0L+S4iX6wiwZ5tysJf8nKK8g28gI+k7SDLY8tgFvvChwu cJWw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id q185si84720wme.1.2019.09.26.02.40.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 26 Sep 2019 02:40:28 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id x8Q9eR1i019032 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 26 Sep 2019 11:40:27 +0200 Received: from md1za8fc.ad001.siemens.net ([139.25.68.180]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x8Q9eRL8024804; Thu, 26 Sep 2019 11:40:27 +0200 Date: Thu, 26 Sep 2019 11:40:26 +0200 From: Henning Schild To: Vijai Kumar K Cc: , , , Subject: Re: Discussion: Base-apt features Message-ID: <20190926114026.182600d1@md1za8fc.ad001.siemens.net> In-Reply-To: <20190925074122.GA12490@lightning> References: <20190925074122.GA12490@lightning> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: iCaHaDg2UZUl Hi Vijai Kumar K, that is a lot of points at once, we should probably go step by step since they are all more complicated than just one bullet-point on a TODO-list. Let me start off with a pragmatic "solution" that is easy to implement and likely to solve a lot or problems at once. Instead of relying on Isar to get you a partial debian mirror, you you just mirror upstream and freeze it for a product. You would end up with a consistent mirror that will contain all you might ever need, all signed with sources etc. A little big ... yes, but easy. If you decide to trust in the availability of https://snapshot.debian.org/ that might also be a path to address some problems you might be seeing today. But yes, we probably still want a partial mirroring mechanism inside Isar. Am Wed, 25 Sep 2019 13:11:22 +0530 schrieb Vijai Kumar K : > Hi All, > > Starting this thread to discuss the base-apt features and limitations. > > Here I am listing down some of the issues/features and possibly the > need for them. > > > 1. Support for adding source packages. > > Currently we have support only for binaries. The corresponding source > files could also be added. It goes even further. You probably want the sources for every binary you install. + all the other binaries coming from those sources + all build deps, again sources and all binaries + recursion of the former We would have to see how big that gets ;). > 2. Support for using password protected keys. > > It is a good practice to have the gpg key protected to have an > additional level of security. Right now ISAR does not have provisions > to use password protected keys. Let us put this aside for now, is is technically just something small on top of signing. > 3. Support for specifying the signing key. > > Right now, the signing mechanism uses the default gpg key of the > system. This is problematic in many ways. Especially for CI. In the > current implementation, eventhough we specify the key, we are not > really using it. Did you not have a patch for that "=yes" bug? Anyways signing is not trivial and we need revisit that anyways. There are package-level signatures and repo signatures. For the partial debian-mirror we just need to take care of the repo-level. All customizations are currently coming out of isar-apt, which is totally unsigned. > 4. Support for adding packages only to base-apt. > > Sometimes, we might need a package to be present in base-apt but not > in the target yet. Things like dev & dbg packages. It would be good > if we have something like BASE_APT_INSTALL which contains the list > which would be populated only in base-apt. I would suggest to have another image to do that. You can do that today already. bulky-image_0.1.bb: require my-image BASE_APT_INSTALL = "strace gdb" IMAGE_INSTALL += BASE_APT_INSTALL Now you build that and ship my-image. I assume that my-image will be able to apt-get from base-apt later. If that pattern works we can probably add an example, support-code and test cases to make that the "supported pattern". > 5. Refactoring code to consolidate reprepro calls. > > Right now, reprepro calls are spread across the build system. Its > dependencies are spread across too(Handling envs like GNUPGHOME, > distributions file etc). My first thought is to have a seperate > module implemented to handle these calls. I think that is not too related. The calls for isar-apt are spread and will probably stay that way, but they should currently not deal with anything around signing. For base-apt i envision a post-process task (like implemented today) as a first implementation. If we decide to build it up on the fly, those reprepro calls, downloads, signing will shift forward in the buildchain and will spread like we have it for isar-apt. Next steps: - try the idea in 4. - implement 1 with revisiting [1] [1] https://groups.google.com/forum/#!searchin/isar-users/base-apt$20caching$20improvements|sort:date/isar-users/_dqKYWUtTa0/velGfVg0BgAJ Henning > Like how > https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/gpg_sign.py > is used for all signing purpose. > > Please add more if you have some features/limitations which needs to > be addressed. > > Thanks, > Vijai Kumar K