From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7204739790011367424 X-Received: by 2002:adf:ef42:0:b0:2c9:1a02:62c7 with SMTP id c2-20020adfef42000000b002c91a0262c7mr81596wrp.6.1677530590674; Mon, 27 Feb 2023 12:43:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6000:a18:b0:2c5:55ca:3a3b with SMTP id co24-20020a0560000a1800b002c555ca3a3bls6383637wrb.1.-pod-prod-gmail; Mon, 27 Feb 2023 12:43:09 -0800 (PST) X-Google-Smtp-Source: AK7set+t1ON1/Hzne5os1C90rxXdzuK7O4bM70BvXrzgN0hMc+HS8VXWE1pXAlCcEzxr1XbuNDmh X-Received: by 2002:adf:dd10:0:b0:2c7:ea8:8afc with SMTP id a16-20020adfdd10000000b002c70ea88afcmr336603wrm.59.1677530589333; Mon, 27 Feb 2023 12:43:09 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1677530589; cv=pass; d=google.com; s=arc-20160816; b=M3qSRWCa0c2V8iGjyoDU+ZEwGRMv6ejwrv2tMClBabv20xQpxt+59FS0Pd4KZV2G4a vwWi6LafCPZOtuJpdRb66lm4+b9yveZwH801lDPtmU0kM1+P+F0A/atJqa0sn/POE6LX MQCbSkk2sdJ5jGtwYTOMsFKCULEVIjf75Wr2bVzG3UfNvTgy1KW9tts9L9H8vE4uR2cO iFulh/k7O/oDGM/9nNJGDwoixhmoG11umU9vA78TmWyUWUwmUaUq+XxstkfGDuCydCVp uXNMy3sNVSmkzsJz7JypIbx2K6Sz5WpRP/hvMTrYkbgsl5ZldEcXInR7XNGirTZBfMP8 +AWQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=W+aQaS/6gl231a3EWxVkbYADgSoRBOqtwQUeO80y7i0=; b=jjfzKpD/f4VgteEWnLiQqcQtioVg1CkhEm7sm7TbiEGW/Ppp0AQAhsC3N8Ljcs7yIh F2s+gRIVvU3EkH3iVzj4B8EuDgHhmC7wLDZQm6StvS1sqOJZ42F4kxNxze32Z8HK3uL3 CQmLVPsVmp10DXClQsVUwKiqPr6I8HzY9pDJDAZYt5oWLpCSud/9HMlUKY6bFpsZlqPH eQL3if4NYBzQcsQETFqMP6zbGL2igj8qprfBQCFXTB2X0Q1e00fhpnMbmeExle/vN6mb Jpa2jaJLt4t/X7P02kwvZ5qcfhh1RIR6Xt9Aa76KUyD4MJGSOpycetiLPoqYbaw92XWn +pjA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=OjY7QmeF; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1a::60a as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2060a.outbound.protection.outlook.com. [2a01:111:f400:fe1a::60a]) by gmr-mx.google.com with ESMTPS id 12-20020a056000156c00b002c6ec127706si339092wrz.0.2023.02.27.12.43.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Feb 2023 12:43:09 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1a::60a as permitted sender) client-ip=2a01:111:f400:fe1a::60a; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=OjY7QmeF; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 2a01:111:f400:fe1a::60a as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=S79j/W/gWWv3gfBxJzyprHEaMh+T/s99eA7XCJYKOmUcvK0IDi4UGa0NsYSVR5eYgHPyzWzQh1K2Q6+egNIsspKhXXymjw5EJ6M5614IxMqe4wOtCLffNo1upFPYjT/yONl+ya+tPYGtD6mfH8SF4mKG3WGmiLfKY+phHhrCUoQ+jmieAMExNalzU0+KK3+FeDFvGVPNbXc3lpfSNPNvCPfPfK38uTREI6rVn6ZCvBfllPoRHAAE+YNsjVFTBuP0Gneeav8UjWmKsU8+g8ewEA9OnX3IIyNg2dioH5Q3zRoiKxPIfCxeTrCONrjPYnDJUiFO1/KAf4T4gz0i6QzhFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W+aQaS/6gl231a3EWxVkbYADgSoRBOqtwQUeO80y7i0=; b=PV0qNCnw4CxkOXNtTMaQTL8IHJZSHYG7dKz0/Iw6wvoRfFgmCq1Ypq+NFc72LMDhGKme5bkINt6PXImr+VE+K/WguN0k7b7pd2f3y69LDw1mrS7hxCalY/JqVTVwlkSd84LqpwXB3IWLWl7g6/etblmg/5BV31Rylsoy+OTmOWOWGONbt9v8gEGWghQMTqTpA9sSe8Id+CgEyDZHGjX0g+pb2ZNuHdmcxrHIA5F2uhr/ghc4ftk8BuW6faYfHoQqzi+Oi/iT+G321rjnfo1k0qlGVPyQis/AV83Qz3bhNrqoRczCQgcUGNkcI7EUCYLlGVlreyE+f7mLjjDkAh8vxw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+aQaS/6gl231a3EWxVkbYADgSoRBOqtwQUeO80y7i0=; b=OjY7QmeFKjmsbGBI5cU7Vd2ag5ZsdydWRzyXEWT9LTXIZANCtrxnXI6CBPe98FqderPsw2aM8N5APLt9zzAaOsBPiRb7oSgBM9eg5JwT5QFc2hschmr63TezQYgKV8ZOPToKoEdTVv8Q1ozBo+Acxnx7Dts+wnnpYaNfpeXotIPyHJMww+4Ny52gBHgQzlnfexoPnj/OFphd1KkPRMUDh8nz35Izp6O9CEbxKiTFC+RK3YVlrGBW0XRyPILZTPrvxCLon8/ygSI1XJL3DNUltlQJy0zDVnS2Wx1qqXhzj0MpeHp1vjGaFMlZTtpZ5v/2ynX7LgU1hdasAoFznxkEHA== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by AM7PR10MB3688.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:20b:138::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6134.30; Mon, 27 Feb 2023 20:43:08 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::ee44:171b:4c40:d42c%5]) with mapi id 15.20.6134.025; Mon, 27 Feb 2023 20:43:08 +0000 Date: Mon, 27 Feb 2023 21:43:03 +0100 From: Henning Schild To: venkata.pyla@toshiba-tsip.com Cc: isar-users@googlegroups.com, felix.moessbauer@siemens.com, roberto.foglietta@gmail.com, jan.kiszka@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: Re: [RFC PATCH] image.bbclass: create separate task for creating reproducible image Message-ID: <20230227214303.5decf5b1@md1za8fc.ad001.siemens.net> In-Reply-To: <20230227075032.28099-1-venkata.pyla@toshiba-tsip.com> References: <20230227075032.28099-1-venkata.pyla@toshiba-tsip.com> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0108.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a8::8) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|AM7PR10MB3688:EE_ X-MS-Office365-Filtering-Correlation-Id: 6b8b2f89-ff7c-4240-13ab-08db19033b90 X-LD-Processed: 38ae3bcd-9579-4fd4-adda-b42e1495d55a,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w0ELB2Fh+v4/NCsMzCfIahdpFi1WdhZfQouvC8A9UYqsUuaU3RU/vfyj+KfI9mElOQgRFz6woQhJGPy+MCxb3ajXGhr77S9Dfy8rgR5moTl6GgTpXCsKiTYGhSYHYj9XzcxuS7N3+f1Zwt5A3S4bEuEd+kIevn8HYQGAlMsDsVHbRBotykpDavr7wuuSdrplDKY5EIMBk7uI+8O2SIWDrMicWRmqAoYOyF7yrkiNTPBqmBF4Rs4W5thjH9dfOysfd6Pfgh4tgTQksrft7n2JbC9eguQU4ZXr+9bLYBxlM9iYHXFUGquSnxsoLJ0GyyUFCVDBSGisIxULizwNuEEmfsb7jmpLEBowQvCfmbj0yoq2QivF3y/tCl9S1MXri+BPq1k2ZTJR8D/jMbTPBudBwvOXqcobfv0rGyAvYDF37Rwu7KYOAp0R9Fb5uLxBxTsBaredVM2YB0LZBMbZ3c++ow7oHZCESchC4cKCUpZuAN4rszg/2QviRAUQXQvizZ4XmCqL63hCe7KEzGeB+kf6vpfy3L8c9APMrzAvjGWLI8TUgQhIXMjNuIgmoAoWPc79Oa+bc7/dnSHm4egL50hCscfpq4NHl1pToy3lYVx3cD6ui0nHfgYgtMXpa9DvBsBaOIidObZImpO03zm9U3grFA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(366004)(346002)(39860400002)(136003)(396003)(451199018)(38100700002)(82960400001)(86362001)(2906002)(44832011)(41300700001)(66946007)(66556008)(66476007)(8676002)(5660300002)(6916009)(4326008)(8936002)(6506007)(1076003)(9686003)(186003)(6512007)(83380400001)(478600001)(316002)(6666004)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Crv8A9kUDxLN/JghN70wvoWIsghYErvJpPzdLtjrdUO/Rzw4HCCyuCcrQaVN?= =?us-ascii?Q?S0Mq+MqE6+OEv3puUltrAUagZ9Vk/o/4CurbQbepHRpjGIjk7Fg6N6cMzMLT?= =?us-ascii?Q?DzIT7Gkj03/MrEQ/sUW9OqeYn09zXeeqcMaE6X9iHsNNqJXe6d2LMR18GOJu?= =?us-ascii?Q?uYOuznVzllkzbHFi1bqHL6nTCg3k3dG2SMoX0qvtvAWrFaA5OF3ivghqb1fY?= =?us-ascii?Q?5NHOkhdPCrJPgQdyb4uiiP8CL2ZL2ymbOIkKT5YyYTCamhtjfmPUp2mWN8dT?= =?us-ascii?Q?X+zArW/9gSk/IQ64EPIUCZabcImBNmUWNozfzju/ODm9nWb4zT5xCM1t8X6+?= =?us-ascii?Q?6OWcpcv2z7Xi3FIsANlqr86w3gLJjDc6YtswGaTWhbn6mRnOd+lprL3ROoe7?= =?us-ascii?Q?I2fjvrlTG4Zy6Znk4gD44FNmsZ047ISdvqAg7NBSfi4kKurSUAGk+j4nfAa/?= =?us-ascii?Q?6ihCf9y6dzGWTzKI3JUCh7Rsm882G5cyQaVC7vTq7OYQ4CY9ABEWEajpLVef?= =?us-ascii?Q?U3OUlHgODbNMKan/WX6cXtXXtASvEFnE4bM8Wj+pYJUz7Bp+7TNcQ7l+MowN?= =?us-ascii?Q?8jtyS33NLruZcyxyCqp82JNAihUxmdmLwXzz/eITm7EcIfs6vB3DUIA0cgyX?= =?us-ascii?Q?4lnEzHCO0aiyapJSbl1pjblOoOzXYC8emunvOy5oajYMxboGscfLDQX54rqj?= =?us-ascii?Q?qY1AgD65xZXW5wVVcjd9fZJXQs5qVSIblrBPtn+y34HUQ8ExOjVq4p8LOP6d?= =?us-ascii?Q?G9Vb23MqGTWiLHFqPkliGRhGvQbJTEkmsuZ/3xaPj7XLpGg0YwUnBAWJ4ZK1?= =?us-ascii?Q?PGwy9L6K8EwbfQIDnXBFNNL70gWr/dsvLlHLY4qebOfdPh09ribBPgxf0Au7?= =?us-ascii?Q?M1p5tDVmh1y+42BG6DbHkAeAgCyAve3FQA0Fvo+6AR0q6g7t2ANiSlMrc6g8?= =?us-ascii?Q?3AzjIL/Xxp0AYu50SlX5nfNBNpaM2ziad/6j9o79sEQbS8RGLrO/7iYAfWiO?= =?us-ascii?Q?wZbTawbdTmDbGlG25GG4NUQt8EEDWe4d04Z4Xd4oBRz7fECGOJUqWzUlxlBm?= =?us-ascii?Q?uxP1Y0b2Y/aU95c+84FVKpBWdarnMWtmOdf4tpLcLGJKpQVKgso2WFQXDhUE?= =?us-ascii?Q?x5OcOiVDJAkKxhk7vfi1jpW0ZZJfxgCXMJGb76VAPJOxujrEXFCJTwEhRlMh?= =?us-ascii?Q?S3oGTX6nobnSVeg6Q1RTQwMgkkIO17NGv0G8aLjKINqFjnVUAcHXTSCr8/I9?= =?us-ascii?Q?bwVL+Gc6rREr5f0NQu5mNboqVq5gByS7mAjrp1fZZFgRb8SUPbU1NmfgDxMN?= =?us-ascii?Q?aYRhobISk+zKdjMcdP0jwhAAXmZHt3W0IZWFV9sHtOuxVRHdsATZ/zy26RVH?= =?us-ascii?Q?fqHQ5JhsVOYGKYIta023RrdinrlzywWWE+BHqe7AwqwW/GL7OqDAcWprmiXu?= =?us-ascii?Q?TjzKddMExZr11s3En7SEA+e2j0DT86DYixvhOl65VDiVSlLewgWkTuhs44bw?= =?us-ascii?Q?ZKu8/I+BHM6UzF12MOTLnRH+/9UXXH8j20Y4dO+QE93Vbm7YZ1W5i7M+cMo9?= =?us-ascii?Q?Y6PU5qW5SU6GRe92UPuGYELZvrLgRsFCjkRawDH3rIP0vnjrfkDFP5XE5h7M?= =?us-ascii?Q?wgHVK0zQSkAWNZRNFgNzm3y2Bcydi0U+fL6EJqlzVDEFdBG5iiSD+SKEd4PT?= =?us-ascii?Q?VK65RQ=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b8b2f89-ff7c-4240-13ab-08db19033b90 X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2023 20:43:07.9982 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: r5mgB8GaIPrZAedHJrmM4+BcQfVlViGFbIqWEecl9xFwxIsNouwTtsjRdLq7z4UeA1ll4c7zeQQH1SE3+WYDwIHIe3vqzii9Tmb4l0N62CM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR10MB3688 X-TUID: Vc3WVRB1zKNx Please do not default cc me on reproducible image topics. I do not care enough and will see it coming or see it on the list. Henning Am Mon, 27 Feb 2023 13:20:32 +0530 schrieb venkata.pyla@toshiba-tsip.com: > From: venkata pyla > > Currently the reproducible fix for setting same file timestamps across > builds is present in the function `do_rootfs_finalize` and this fix > may be skipped for the tasks that are added after this task or > `do_rootfs` for e.g. in one of the child project(CIP) for swupdate > target it adds additional task `do_generate_image_uuid` that is added > after `do_rootfs` this task is updating some contents in the rootfs > which is skipping the reproducible fix was applied in > `do_rootfs_finalize`. > > For this reason a separate task `do_image_make_reproducible` is > created which should call before the final target image is created > [tar, wic, squashfs] so that it applies reproducible fixes just > before image creation. > > RFC: > In this patch I tried below dependency[1] for the function to solve > the above problem, it works now but I am not sure whether it guarantee > always this task `do_image_make_reproducible` will run just before > `do_image_[tar,wic,squashfs]` tasks or does it need more changes to > make it guarantee, any suggestions? > > [1] addtask image_make_reprodcible after do_rootfs before do_image > > Signed-off-by: venkata pyla > --- > meta/classes/image.bbclass | 33 ++++++++++++++++++++------------- > 1 file changed, 20 insertions(+), 13 deletions(-) > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index ef7d5a2..c3a3d45 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -446,19 +446,6 @@ do_rootfs_finalize() { > fi > EOSUDO > > - # Set same time-stamps to the newly generated file/folders in the > - # rootfs image for the purpose of reproducible builds. > - if [ -n "${SOURCE_DATE_EPOCH}" ]; then > - fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > - if sudo find ${ROOTFSDIR} -newermt "$(date > -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ > - -printf "%y %p\n" -exec touch '{}' -h > -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then > - if [ -e "$fn" ]; then > - bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of > $(cat "$fn" | wc -l) files for image reproducibly." \ > - "List of files modified can be found in: > .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > - fi > - fi > - fi > - > } > do_rootfs_finalize[network] = "${TASK_USE_SUDO}" > addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess > @@ -502,3 +489,23 @@ do_rootfs_quality_check() { > do_rootfs_quality_check[network] = "${TASK_USE_SUDO}" > > addtask rootfs_quality_check after do_rootfs_finalize before > do_rootfs + > +# Run this task just before final image creation of different image > types +# i.e do_image_tar, do_image_squashfs, do_image_wic etc. > +# to avoid modification of image contents that leads to > non-reproducible +# image > +do_image_make_reproducible() { > + # Set same time-stamps to the newly generated file/folders in the > + # rootfs image for the purpose of reproducible builds. > + if [ -n "${SOURCE_DATE_EPOCH}" ]; then > + fn="${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > + if sudo find ${ROOTFSDIR} -newermt "$(date > -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d %H:%M:%S')" \ > + -printf "%y %p\n" -exec touch '{}' -h > -d@${SOURCE_DATE_EPOCH} ';' | egrep ^f >"$fn"; then > + if [ -e "$fn" ]; then > + bbwarn "modified timestamp (${SOURCE_DATE_EPOCH}) of > $(cat "$fn" | wc -l) files for image reproducibly." \ > + "List of files modified can be found in: > .${DEPLOY_DIR_IMAGE}/files.modified_timestamps" > + fi > + fi > + fi > +} > +addtask image_make_reprodcible after do_rootfs before do_image