From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7197029299818332160 X-Received: by 2002:ac2:5195:0:b0:4b6:f30d:4de1 with SMTP id u21-20020ac25195000000b004b6f30d4de1mr2670018lfi.33.1676033289837; Fri, 10 Feb 2023 04:48:09 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:19a9:b0:290:6183:37b4 with SMTP id bx41-20020a05651c19a900b00290618337b4ls880723ljb.0.-pod-prod-gmail; Fri, 10 Feb 2023 04:48:08 -0800 (PST) X-Google-Smtp-Source: AK7set/inmxZ0DyZrbNscKlqUhLtlFIZhs4V3jROWS5GVowd/v4cemEi7JZdrrxY9+oVFSMcOf7P X-Received: by 2002:a2e:994c:0:b0:293:2771:4678 with SMTP id r12-20020a2e994c000000b0029327714678mr2813498ljj.38.1676033288288; Fri, 10 Feb 2023 04:48:08 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1676033288; cv=pass; d=google.com; s=arc-20160816; b=lBXLXRlRBeXhvD18F86SQk3BfinTHEb8ZnTG0zyJyaNmuvUMh1OJfPoHLqV8/nogLs c0FwxJ2a0aNr7nLYBINu/FCEGkwLpFF12V2Unff0nOXB2+RGfUo2hH7vN4i2gUIazQPq TJYo4UpWd6JtagtzKWc8mlDUUAFm9QmKMjFsIw4IJLCXJMl+nkXm0bJH9DuXSY0/TDES bnowLphlH2+59SdFAT2UilA8Gl9RV9ffXDY3S0ohUklmK6H16okc2vZYhiLzGkxV2dR4 udTakqoDpKbjKIcozZzcz139JumgkbyBJkKqGVeSj4ba2ESX4abs/OXIpv71y1Lk5mrY JcrA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=DXzngGlD/HpurEVgsUxKFzIEQTjuDfqSHhi6fdr0xGg=; b=NQ/3euqLZ6YRELgLYMSBVT1kGl/dzAATU0rLMiJ/rm3EXjr95R5ckC5RLJ+Wo50puy 3N9c8bwMTzv15Hq3LQ79tuBfLk3plJ0Ce41HaOJK+JTzBTXFNGwP3Gx36sVw1d7R6lrR Uy38D8mJcsA4Sfuvcxm0ugsSAvwfEFtodZnu0+3sp0YR5kRooXa/HQd595frci38LUu3 lPJ0Z2h0UtZtHi6+5oybP67l3ZyuOIrc8pz0hZGhVGscurOSFCJVEJH3yVWFRyfUGdKw DhKCrjLVkIJmGu+NVM6CIOjCYw3s7FXyTbcDMsEfz01oFMbhoMNTEhX8x0KBmhAQ3cWg hVMQ== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=MW+KWnt8; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1f::600 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01on0600.outbound.protection.outlook.com. [2a01:111:f400:fe1f::600]) by gmr-mx.google.com with ESMTPS id l6-20020a2ea806000000b0028ffa3d673asi210883ljq.3.2023.02.10.04.48.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 10 Feb 2023 04:48:08 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1f::600 as permitted sender) client-ip=2a01:111:f400:fe1f::600; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=MW+KWnt8; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1f::600 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Vvd9nlm5JCksYwoVATbJSLR5JOoAJ8wmV1XZHkbH7w/tWwkeeF7eZy2nGGDLp6JxKepgZWtsQ+QVKXl0XIg2XMVsgiuwgJcKvy06OuhWgvvVOgRmi7EcVE5fQm9PfEoUQZDCs3nEpG2OkPZsOdP7WpPzw/cnDUQ/LtUfWWZx8tDVldZDWR22xVVgRSRjCufDEk8Z227SpNT5qRvt+vGDWTFUqo6pccruUxm25BwSbtN5xzmQM7wMGwdhBrgVysnG+JeUmWyomKOXSAHLz/cjObnwWyujbP7bqo0EUAi+GTGiJxXdDOnDNmmM5PsbxS27MCRj3M7uy4Tl4mmuUVEO1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DXzngGlD/HpurEVgsUxKFzIEQTjuDfqSHhi6fdr0xGg=; b=CP6czyrZwSEi3wiTRqwbhdKYoLkOVH+NNJB1BjFxzgD11OzkW6PWZ0Q40fJwTDt11UenmuWneD2aTSpZ/GDOWyS343Ne2Qf0fn18RM+xzBSjbiya5moMpw5CZGn07AnLhQRmS8FxVbtTHcywro6C6B6kgsDyHOl6m8jDpnTVuYZ2Pdd1HuxI0eXbPBZ0sFwj6qVxqs0Pag918BFcp+Zt+axsA7CMWYQCs/yzmBWjPAYr0CCkMpvXZ4Yk7Tzzwlp9Y2PVLucx9z61ix1MfqBrzeV3vYUMRCNb9HqIXdISG7oYIW0OxdWvJg1UGH9FGJaN4xd6eH+WR3GFudatyLtfnA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DXzngGlD/HpurEVgsUxKFzIEQTjuDfqSHhi6fdr0xGg=; b=MW+KWnt8bnnKn1sxRxiXxbMeY+dw6dkxTa0o5/D73cKMAa0Squ8At7kWapqoqNzkLgjtwXP8/RP0dKVlEEqqIz7M1X+gKJfF1R5qy6m3yflj8dU86TD4MT4IgLEzavks1/2yeqyO1jO5YeqdHK3edv2dEc/GEYrOY6w2GaPNvS1mDFY7hgmlucKOHfitB8rrNiaULopUwxwJnmGK3D3qBxv4vKtL7+kEBEXFiV/oKb24W/ppvwzHmN4W+t4XFLueEDl2hur3cAzhYMZFozJ/GD67YZFielceVixLvsYulrn+k2R5W5T6PH1faq+/eUw3j2g/rkdVN1V+Q3zqvr24Xg== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by PAVPR10MB7257.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:310::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6086.19; Fri, 10 Feb 2023 12:48:06 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c%6]) with mapi id 15.20.6086.020; Fri, 10 Feb 2023 12:48:06 +0000 Date: Fri, 10 Feb 2023 13:48:03 +0100 From: Henning Schild To: roberto.foglietta@linuxteam.org Cc: isar-users@googlegroups.com, roberto.foglietta@gmail.com Subject: Re: [PATCH v4] deb-dl-dir class rework to use faster ln -P or fallback to cp Message-ID: <20230210134803.00fe46b3@md1za8fc.ad001.siemens.net> In-Reply-To: <20230210110908.1507520-1-roberto.foglietta@linuxteam.org> References: <20230210110908.1507520-1-roberto.foglietta@linuxteam.org> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR2P281CA0114.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:9d::6) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|PAVPR10MB7257:EE_ X-MS-Office365-Filtering-Correlation-Id: 747509e9-7aec-4f1f-92e7-08db0b650e26 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Jqt589eqUwkCvaBVAglLgHHih7qouJgkl4YuuQk6X0y/PmUkAGRXn56FJ5/74R8A//ek9kUkdxAT/OQVtkirhcuIDDCoeQlbC08rFXkWb/3XsEOHJ3lsgo8FYLMFzMHGEQH4Yrno2OeBDPit435okuDBK+UERJAXzBOMm/elkRyxWHBHz1HXDKRzdGve0WBshkGMaTkM009tormkYPTWD/suwW2Y5NgrAwr22/hVWn9oPhWdF+TnnMH6E2penbCEnMk3a2ajTbW7fKaPgYw0yLA9xJXmwFDT3ImV1yYDwUuDS3OduZFEjT0Lqroflqc3XfLzOjjFuRX0UMkDSzKeWhJQIA8Hx56mWuGSu+l8Ppgh8LwOc1uiyANdbOckbLvq7n6K6izhvhW5IM1+vAxbFIj8uZUuP+9gwY9n2rTh2oSd1ACh6+DN5Z8UbudKz1K4G1Gdv3g/pwLVr4PO88GGsrClf54Esfyf3xogit1drUb7Bnnw7P6B1uGo4cSP57LenYXuvLY+tuhDwukY1UyNQ1BghyIcGSNBzY4NU+K59lo6fyZs2FTNaFmsgy/0/wKRJMYC9qJQxB91VGnsw7LjWeylMKTbxUOc1OY7bzG+ALf3DRxyrwx39WLAAzIf0jhbH/vhuaUg+UQHUNmqrPIdOg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230025)(4636009)(136003)(396003)(366004)(39860400002)(346002)(376002)(451199018)(186003)(26005)(316002)(1076003)(6506007)(6666004)(82960400001)(38100700002)(2906002)(86362001)(6512007)(9686003)(478600001)(6486002)(44832011)(5660300002)(83380400001)(8676002)(8936002)(66476007)(4326008)(66556008)(66946007)(6916009)(41300700001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ll08kjD8CBMXWv7NmYAwvyiVeRhUpeRRLGRT6n4sfRwKsVf9Tq18qbTDt42n?= =?us-ascii?Q?B8ClZ2alh+IZx4dJRJVtu3OfvF9cwJKplAXvfJDHLuezrvIIB+WhoOTwjwiH?= =?us-ascii?Q?Nz3KnHNrbbs3MTHbMYcYYOeI5D+3ZNZgi00wgsLVXw+4oL/Wh6IWavsJE2q6?= =?us-ascii?Q?+EmHb6rG+9JUatwPBSQxjywvkjm23ULHJSZmKZXScz7Nk9Wohf9jJUg6sTLR?= =?us-ascii?Q?26ofu++seUCjiz1sGMjPP+PNbJhwlidBuRP1U6cWHDeqUsjCbH7zM47iJvXj?= =?us-ascii?Q?T9OmGPi1n+q+NSTWjtbIc6BTf/Vs0sj0bUopd1ckV7B33RW84aO0HTtucOMA?= =?us-ascii?Q?RF4jNN7k5zh9oP6RkIEFB1F9Bziz/t3GwqTcUr9ceQi3hT4PeYy0ObGJOjKj?= =?us-ascii?Q?yVPkUrhCuuLMjiQ1HUUlb50BnFzg1jXWJUsL8+oQUZGeWdaFPYWuVu+W1cG0?= =?us-ascii?Q?zPGzbYFxe120H9rjaFmK0JM8YbKi5eNrRNNAAARXc438eXg7GkDVfNglMs9x?= =?us-ascii?Q?frKX+wPg/XsRskTDdm2hUci/JshtBb1Xnwd5K8gaAT/K8FZXLeSt2wSNJIur?= =?us-ascii?Q?kQU4cuZdFJj1+wxinayc7KXXCWBr+QEox+hCON3MTuP3RcnZf0gi0Y9uVQv9?= =?us-ascii?Q?L6sLt/U6snjhHRBUYZSfnzKPiwjmbjHaObHN3ZAQjPgwWlJOaFkSBaVrZX/+?= =?us-ascii?Q?sd5H4w7gTXXv9nV5Y1zDdac6rMYAS/yM8pGI72SxX4ALRWUkcIo8/DSP/7sr?= =?us-ascii?Q?1UPRX1K8nDvJvodrfB0POyIvKigbrCnKglppPr1sGBy2ZFdOW8UNEwe/oQJw?= =?us-ascii?Q?Wlp4EXqDAWS0bUjyx9DkjbVne0Ne06mrxT0ub9WfwNVoHiQcc5dbJ+SFejO6?= =?us-ascii?Q?SHByxKjBuljGkM3347GqR43UKnGL7mJ20sJi0//QKJ32pjm3RDdRLPRyfJ1Z?= =?us-ascii?Q?oPP0Kly4SJrJezlBFzFW/7OU75vcNKoVhTN5f/duoUrLsBCaq9vv2yBj2Kn8?= =?us-ascii?Q?vnxXSYoAbZN9KAe0YoNrZsB1Fgw6KLZB4hc4Bpd4/xjt957+woIxaJDPVYJh?= =?us-ascii?Q?qx5b9b3y7n0Fl75PDdtLxoC4aR2msD/8e6OU0fy+HgnPNS25Ure/hFzEIxDR?= =?us-ascii?Q?/U+el5ujN2tFoBH3OgQsmUT6EhfIjri+5S4TRsRzz7NHi1WbUV2XhkCEB/Ns?= =?us-ascii?Q?F+abSKkxXplresBSb3dI8QfIXVHuHUCy3dQf2uiCZLXZ7ejk3pGzMmfnbixt?= =?us-ascii?Q?W9oG4kK+ktIvG7RZLfJcnmPI3q6jIeGouGOVLT5oN0fmyiuNpKlNOvScbGcN?= =?us-ascii?Q?do2k6JWz6kcoSBGzvduAKfRPby50Rakwg1PX6loAfvQGtzpxmLz9JDTPSVgB?= =?us-ascii?Q?iSOqnCFjgpI8F+1RKduR0aUpzW7ggGfpuBvYjtMaAgcy9f5ai7ffXnIS6Azv?= =?us-ascii?Q?kPqw7jqkLoAbHAkT19WP6DxkF7tOcdDyqL53aHWipN3YicLrTIrBD/QSsCxc?= =?us-ascii?Q?+Cs9wHPex14TP1ZZrwsC21E2BD9QBPjC1hOeRwl2iJGxg17/35YyyNs9+fEB?= =?us-ascii?Q?gQu5ZcBdgtLkw+z9/BJffM4vgZ0fcpJmPbHZNccoFo66ucRi3rcY9ZaOS/cP?= =?us-ascii?Q?28wPoH+lJQQmWuvNkaa67fs=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 747509e9-7aec-4f1f-92e7-08db0b650e26 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 12:48:06.2099 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: onXq4uF+tU3ig/WvyXZPPaEL81pRVYSghe3LTQs0Exbbd81uVLdJVjgGfkIzelDTmAaG7tSge9G3OpqhRvoBqGzkdcnueb3xSDJlIE3Prro= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR10MB7257 X-TUID: 1Qq+Tk/HOg8g Am Fri, 10 Feb 2023 12:09:08 +0100 schrieb roberto.foglietta@linuxteam.org: > From: "Roberto A. Foglietta" > > deb-dl-dir, feature: faster when using ln -P otherwise fallback to cp > > The original class functions deb_dl_dir_import/export were using cp to > copy debian package to the target rootfs but this approach is quite > slow while using hard link does not work if the destination and > source dirs are not lying on the same filesystem. Thus, ln -P should > fallback to cp when it does not work (which is different from > complaining on stderr). > > Moreover, these two functions have been reworked to reach a straight > forward and more compact form. In particular, export function was > using bashism to do some kind of comparison which after all is useless > because copying back without overwriting just fulfills that part. > > More rework using sudo in a different way plus a corner case > addressingi, in case the spia file exists for some other reasons. > > Rebased on the current next and bugfix about checking the destination > not the source file. > > Signed-off-by: Roberto A. Foglietta > --- > meta/classes/deb-dl-dir.bbclass | 53 > +++++++++++++++++++-------------- 1 file changed, 30 insertions(+), > 23 deletions(-) > > diff --git a/meta/classes/deb-dl-dir.bbclass > b/meta/classes/deb-dl-dir.bbclass index 7db25251..2a3c7508 100644 > --- a/meta/classes/deb-dl-dir.bbclass > +++ b/meta/classes/deb-dl-dir.bbclass > @@ -78,39 +78,46 @@ debsrc_download() { > > deb_dl_dir_import() { > export pc="${DEBDIR}/${2}" > - export rootfs="${1}" > - sudo mkdir -p "${rootfs}"/var/cache/apt/archives/ > + export sc="${1}/var/cache/apt/archives/" > + sudo mkdir -p "${sc}" > [ ! -d "${pc}" ] && return 0 > - flock -s "${pc}".lock -c ' > + export tf=$(cd "${pc}"; ls -1 *.deb | head -n1) > + [ ! -e "${pc}/${tf}" ] && return 0 > + flock -Fs "${pc}".lock sudo -Es << 'EOFSUDO' Just curious ... what would EOFSUDO stand for? We have EOSUDO in many (all?) other places and every author can bring in their own style. But EndOfSudo is clear to me ... EndOfFileSudo seems weird. But maybe whoever used EOSUDO first meant something totally different with the acronym. And it does not mean EndOfSudo ... Henning > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x > > - sudo find "${pc}" -type f -iname "*\.deb" -exec \ > - ln -Pf -t "${rootfs}"/var/cache/apt/archives/ {} + > - ' > + rm -f "${sc}/${tf}" > + ln -Pf -t "${sc}" "${pc}/${tf}" 2>/dev/null ||: > + if [ -r "${sc}/${tf}" ]; then > + find "${pc}" -type f -iname "*\.deb" -exec \ > + ln -Pf -t "${sc}" {} + > + else > + find "${pc}" -type f -iname "*\.deb" -exec \ > + cp -np owner --reflink=auto -t "${sc}" {} + > + fi > +EOFSUDO > } > > deb_dl_dir_export() { > export pc="${DEBDIR}/${2}" > - export rootfs="${1}" > + export sc="${1}/var/cache/apt/archives/" > mkdir -p "${pc}" > - flock "${pc}".lock -c ' > + export tf=$(cd "${sc}"; ls -1 *.deb | head -n1) > + [ ! -e "${sc}/${tf}" ] && return 0 > + flock -F "${pc}".lock sudo -Es << 'EOFSUDO' > set -e > printenv | grep -q BB_VERBOSE_LOGS && set -x > > - find "${rootfs}"/var/cache/apt/archives/ \ > - -maxdepth 1 -type f -iname '*\.deb' |\ > - while read p; do > - # skip files from a previous export > - [ -f "${pc}/${p##*/}" ] && continue > - # can not reuse bitbake function here, this is basically > - # "repo_contains_package" > - package=$(find "${REPO_ISAR_DIR}"/"${DISTRO}" -name > ${p##*/}) > - if [ -n "$package" ]; then > - cmp --silent "$package" "$p" && continue > - fi > - sudo ln -Pf "${p}" "${pc}" > - done > - sudo chown -R $(id -u):$(id -g) "${pc}" > - ' > + rm -f "${pc}/${tf}" > + ln -Pf -t "${pc}" "${sc}/${tf}" 2>/dev/null ||: > + if [ -r "${pc}/${tf}" ]; then > + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > + -exec ln -P -t "${pc}" {} + 2>/dev/null ||: > + else > + find "${sc}" -maxdepth 1 -type f -iname '*\.deb' \ > + -exec cp -n --reflink=auto -t "${pc}" {} + > + fi > + chown -R $(id -u):$(id -g) "${pc}" > +EOFSUDO > }