From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Fri, 19 Jul 2024 18:38:54 +0200 X-Sieve: CMU Sieve 2.4 Received: from mail-wr1-f58.google.com (mail-wr1-f58.google.com [209.85.221.58]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 46JGcqt4002628 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 19 Jul 2024 18:38:52 +0200 Received: by mail-wr1-f58.google.com with SMTP id ffacd0b85a97d-3685b7bff66sf157415f8f.3 for ; Fri, 19 Jul 2024 09:38:52 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1721407127; cv=pass; d=google.com; s=arc-20160816; b=0k8bs+uPT3lr035vp0ngHKMjQV7GsprgD2a4SZfBcr2x4yhaO+L+Me46BfMnqqI/WP IMhOtla1xSqrydkxymT0yks64pdGmCMorP5w2tqygWGIHfa8/oViVpguzJegBQS0ACj8 w8LplmAG/ynHkE1iLECksOxQyfsBljys7dLOmhrHbdfLWL924C2CBFcsJzAiBcnodHa4 OSjzRpAF2q0SLufpaDz2gSSsSfcIIhmmS5DN0mzDOwXZG3xutunlmhrNExjfHxoXJzhz TxOmR0MjwKaf2o0+M2EDnfUbUT9XfGOvUbj8ObQHZ6ZjJ3K9WzxbT5+wwaAqnrgceUfc bh2g== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to:feedback-id:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=wVGdmxEWXjjK36JufY9jKaOTGHBl3LdsJRx/k6m5zIg=; fh=1rt6su4Pf5jHuVo98zF+slxyl1jqIURSG8GxGQd/sUw=; b=Yc2h+yTOddxNxDV4n11LQhzcbNQ1Uku8BUQDfx70V3x5Y6s6/R1xUeFhMDO6aTAog7 FzT0+JdfYdVCB3OgOqbDQXy7L2R2do6VvMpS6GCMvxbpIZvcwGvqe5bJUoE9BAtodpe6 FHeWnLKe0r9bEGswfE/3oIMdNXgZUQxB1lCQQpuq5/0MBKuxdWCw7PQ1SmJW7M4NfJOo rQXqXb/G+R3yPf4/soYpmRcIURWgtuvzzCfJ6y4clDCVchA5jmt90U6yX8sEq+DKZSNF EsAoo1qKPRVbsq+O6gTrL0mCQLYabl8w+oWONt0WWnNc7a9QmBxRzrz1S8sSoB+CS2g1 67MA==; darn=isar-build.org ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=hVVBMGX6; spf=pass (google.com: domain of fm-294854-20240719163843ba90b9cba9133cc524-0oqczr@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-294854-20240719163843ba90b9cba9133cc524-0OQCZR@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1721407127; x=1722011927; darn=isar-build.org; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=wVGdmxEWXjjK36JufY9jKaOTGHBl3LdsJRx/k6m5zIg=; b=M15yvmFLyXqaCv+6wMuRi2CcU3qBHnkVaVFD98DpdmaCXYLwP7yT0dMgZmm9bBYIQY tlp+3/H4VfaA/pfErcWiUDXy3L5ZZ3lhyuYGrK+f4ehSGs/dW1DqagRAnHUuk5pYe7iS c3bEFhlYiOwvpvIsjLZgd8Lo8BEiqb96hwIue+DsE8rEHH/XltUXNUn/HTcgF7mM+w97 O+3pm7nuHhTKmcz6ZpvmUoTjz1quVWShqG/AVhWq1PNLqYx5FHZ+7fpCEY418r7qwrqd 3qhFfxj4u0SH2sp/whjeimLDq09RTE0DkgySUk5EFJC17WfpJROO2QFvqhJcmcoURD8J 1xNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721407127; x=1722011927; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence:reply-to :x-original-authentication-results:x-original-sender:feedback-id :mime-version:message-id:date:subject:cc:to:from:x-beenthere :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=wVGdmxEWXjjK36JufY9jKaOTGHBl3LdsJRx/k6m5zIg=; b=iv3KGf05np0ucdGJnjUOzSsBKLxyQFoO1SHPHI2HodyYKdLJHoz+vqiHB+nC7lElfd jZlg5VVmSMv8t3c+jKa1w8iqmR+R3iMbxO7IqH9TP+ZxgWPQaDb0+LaLfOJ3F3pirnxI Etf88cPjceF0mmkEF/oEDeRKHC8Z2l5ICY1+9IX3B1tNLcQ47cVeFLKERb3PMTYalWRK 0WflaPInsVZgo5eIT5zo8kRmUj9dwBDybi6Jp7HJ7B4OE5DH0sIvCAk+NpwuKrNwWnqG WVlmbwdIa3d+GhW/M5GQgkkZaDxbniIxMymmp+54f1E2m0Wa816oMjDz3i+q4a2IpYfy 3j/w== X-Forwarded-Encrypted: i=2; AJvYcCWeboif8oYiJ5n32+Nuhw1SQVpwK7Z0JdWA2YeKEnao+Otzzmy+4g0MgrQNeEXeAabw7H609bDk2QT4gsuNDelpvss= X-Gm-Message-State: AOJu0YxxYJGB4ilH27DcrKlnKEAfEk6hsNQzOmzkT/wNvJPsC6DuXTAU jf1oKv8s98TXQryM4qaqkoEsR6C/2TjJbvT3VEI0O2kgWb7xUXkh X-Google-Smtp-Source: AGHT+IG3hIjM5Knv6ww8Ne3RUIcB209uCwCQST6DIfckLP/qB7lzPFrq8xRZyt53vuXalQpDaZf4Zw== X-Received: by 2002:a05:600c:1d0e:b0:426:6fc0:5910 with SMTP id 5b1f17b1804b1-427daa29d80mr297805e9.1.1721407126518; Fri, 19 Jul 2024 09:38:46 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:3b8b:b0:426:64c1:8974 with SMTP id 5b1f17b1804b1-427c815639als9954445e9.0.-pod-prod-01-eu; Fri, 19 Jul 2024 09:38:44 -0700 (PDT) X-Received: by 2002:a05:600c:cc6:b0:426:6f48:2dad with SMTP id 5b1f17b1804b1-427daa67d53mr323425e9.35.1721407124421; Fri, 19 Jul 2024 09:38:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1721407124; cv=none; d=google.com; s=arc-20160816; b=Vj76Cx+bRiFCEcpIAJz5ZU8uAOwJcWTCcbmF/vSj2AB6eivmIYW1cAAL+sPMhkJntZ ZNMLP6mmy1SLRrsFzcb+UgsVIYgKhS7e93vk/v4UkzMXvECHPAtm/nDMFGZDvALhdGFX ZhoDQsJ08AYEtm4JEEAk9qH5Zyba6+zS8lXNvPwjCQtmXXSyGsZzLUJjfW5bP9zEXbpx jedascG4QF/rI2qUrt+cpZl+GqtYHGsH6nSg8sbQtYc8bRzGg2Aq3wmm2iGJEwpnJgJz CCWCoaujgpKB1Pevn5RQf1gZ9/qTFFivMWqr9SlwXNFWBZpPpiE4t7E63od3u+YzHk87 rBlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=feedback-id:content-transfer-encoding:mime-version:message-id:date :subject:cc:to:from:dkim-signature; bh=ZwcYfVTuBZdjYZAYZHKneiJFNmDCMmetpeNI7YKwhQ8=; fh=sG8mVX6bNXJXg1RhSNhryk9YKHryCWWF37H72hfFhEU=; b=YzOUtx7rVTZN/KiWfJaH7yb9lR+uARNcdhIHt3LVG5z8NPa7ZKvmWKxet8KHb9bFau wRlYnswyKtkU6MaoB1w+OfA0hO8NRtl2M8n7Ha84aryO6P0tBK6VkFBKQEQ7TsGwb1ZA r9q9gi83S8iNINwHC0i6dCGIli3DPZ87D6p7azqnJ8giZ20YEhBqj7G/SYJmhqgM6hw+ HYFFBW5kJb3olWpZdEHpZH9sYmMP1uANgqnYyUdrQOugtNyQw5xXc9DOeRIgbsfSa1uS vTr380XjeCL97iYPcd6J7PX1Hzwveu3Sjx2me8flTPYN32f6ztwXYIAlvO0Za6djRikA sgNQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=hVVBMGX6; spf=pass (google.com: domain of fm-294854-20240719163843ba90b9cba9133cc524-0oqczr@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-294854-20240719163843ba90b9cba9133cc524-0OQCZR@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com Received: from mta-65-227.siemens.flowmailer.net (mta-65-227.siemens.flowmailer.net. [185.136.65.227]) by gmr-mx.google.com with ESMTPS id 5b1f17b1804b1-427d2911f57si1635545e9.0.2024.07.19.09.38.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 19 Jul 2024 09:38:44 -0700 (PDT) Received-SPF: pass (google.com: domain of fm-294854-20240719163843ba90b9cba9133cc524-0oqczr@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) client-ip=185.136.65.227; Received: by mta-65-227.siemens.flowmailer.net with ESMTPSA id 20240719163843ba90b9cba9133cc524 for ; Fri, 19 Jul 2024 18:38:44 +0200 From: "'Jan Kiszka' via isar-users" To: isar-users Cc: Silvano Cirujano-Cuesta , Benedikt Niedermayr , Felix Moessbauer Subject: [PATCH v4 0/5] Introduce container fetcher and pre-loader Date: Fri, 19 Jul 2024 18:38:38 +0200 Message-ID: MIME-Version: 1.0 X-Flowmailer-Platform: Siemens Feedback-ID: 519:519-294854:519-21489:flowmailer X-Original-Sender: jan.kiszka@siemens.com X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=fm2 header.b=hVVBMGX6; spf=pass (google.com: domain of fm-294854-20240719163843ba90b9cba9133cc524-0oqczr@rts-flowmailer.siemens.com designates 185.136.65.227 as permitted sender) smtp.mailfrom=fm-294854-20240719163843ba90b9cba9133cc524-0OQCZR@rts-flowmailer.siemens.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=siemens.com X-Original-From: Jan Kiszka Reply-To: Jan Kiszka Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 8z1yNv1/R5U/ We noticed internally that there are at least 3 implementations for solving the task of pre-loading container images into isar-built base systems, may it be that the images are not publicly available or the device is only poorly connected, if at all. This series tries to solve the task generically - hopefully. Changes in v4 - fetch per architecture, no longer full multi-arch - directly archive and compress in download task - use hard links where possible to reduce disk usage - always use zstd, remove configurability Changes in v3: - do not compress container deb packages [Benedikt] - fix various spelling issues in the doc [Adriaan, Benedikt] Changes in v2: - switch to open-coded unpack before loading - switch to zstd as default compression - use default compression settings from bitbake.conf - add support for delete-after-load (opt-in) - factor out fetched container installation into separate task (should make self-built container packaging easier) - use BPN instead of PN - add "Requires" to systemd service - use PACKAGE_ARCH to select container arch (rather than DISTRO_ARCH) - add CI test - add documentation Jan Jan Kiszka (5): Introduce fetcher from container registries container-loader: Introduce helper to load container images into local registry meta-isar: Add demo packages for installing prebuilt containers ci: Add test cases for container fetching and loading doc: Describe how to use the container fetcher and loader doc/user_manual.md | 58 +++++++++++++ kas/package/Kconfig | 19 ++++ kas/package/pkg_prebuilt-docker-img.yaml | 9 ++ kas/package/pkg_prebuilt-podman-img.yaml | 9 ++ .../prebuilt-docker-img_0.1.bb | 12 +++ .../prebuilt-podman-img_0.1.bb | 10 +++ .../recipes-core/images/isar-image-ci.bb | 2 + meta/classes/dpkg-base.bbclass | 6 ++ meta/lib/container_fetcher.py | 86 +++++++++++++++++++ .../container-loader/container-loader.inc | 73 ++++++++++++++++ .../container-loader/docker-loader.inc | 10 +++ .../files/container-loader.service.tmpl | 12 +++ .../files/container-loader.sh.tmpl | 18 ++++ .../container-loader/podman-loader.inc | 10 +++ testsuite/citest.py | 21 +++++ 15 files changed, 355 insertions(+) create mode 100644 kas/package/pkg_prebuilt-docker-img.yaml create mode 100644 kas/package/pkg_prebuilt-podman-img.yaml create mode 100644 meta-isar/recipes-app/prebuilt-container/prebuilt-docker-img_0.1.bb create mode 100644 meta-isar/recipes-app/prebuilt-container/prebuilt-podman-img_0.1.bb create mode 100644 meta/lib/container_fetcher.py create mode 100644 meta/recipes-support/container-loader/container-loader.inc create mode 100644 meta/recipes-support/container-loader/docker-loader.inc create mode 100644 meta/recipes-support/container-loader/files/container-loader.service.tmpl create mode 100755 meta/recipes-support/container-loader/files/container-loader.sh.tmpl create mode 100644 meta/recipes-support/container-loader/podman-loader.inc -- 2.43.0 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/cover.1721407122.git.jan.kiszka%40siemens.com.