From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7184069459095584768 X-Received: by 2002:a2e:9607:0:b0:27f:c51a:bc15 with SMTP id v7-20020a2e9607000000b0027fc51abc15mr1483974ljh.443.1672906759873; Thu, 05 Jan 2023 00:19:19 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:4e8a:0:b0:4c8:8384:83f3 with SMTP id o10-20020ac24e8a000000b004c8838483f3ls1778398lfr.3.-pod-prod-gmail; Thu, 05 Jan 2023 00:19:18 -0800 (PST) X-Google-Smtp-Source: AMrXdXtzXhxw0tkyPvDXIUXF2k7cupVLuI1gxrlHwWvzWgzxqp6Cl3GQsavv8i7PDyz3dG7f/itR X-Received: by 2002:a05:6512:22d3:b0:4b7:66:8231 with SMTP id g19-20020a05651222d300b004b700668231mr16305417lfu.58.1672906758515; Thu, 05 Jan 2023 00:19:18 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1672906758; cv=pass; d=google.com; s=arc-20160816; b=D4olls+pItYHsQe0fqb/9WrPgLiS70DFPtlM6JUnu4lC4zV37BTGZkL1cK+tEMzz98 FCo0eAsKvCNkq9n3ZMdqo2G1c7u5mKaRWWwX947W49yEi7AQvC8bR28iP3G2+4URlZUI Ff+/bvcTtE65dK2kR+jNmJD5ylC/tc55L9RtThIzXBjfVjXrgDXhpNJbyEtj0N4X7bXM wCD5DLVuFKyWn6YU7RkFJNyYobjwe1JQ0BpVtB1ZC1tKUom67OWsi1ZKph3pKFQCUDQ1 bUniATH+BrcVFk4HArwJG2U2FXmtw1sozGMzf1x1dcYAt8hZcrw26rQ4EwHJCsnCiS4U HfwQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=0p1olQpyP5dd91Eel7i6RNQkH2EzXpbabk5TefYpWLo=; b=sIyYp4+7bcbXeKcsKN4ZVU775n1f0WYDRcEltuB+hXFSZmr7nia0fB9RAy1koY7llv oSZ35HaAXSDbEJx5U0nIpc55WH86kwH3AlHNm6qh/ulm0fEQF59iGxYvBrlHLEL6pQKU L2vkx7xPUYVSf6YaQ243w4k3fViFloGjtC8xPMIDXh/S/2LM1uUmZ0Cs744mfGJVMspY tqCQPyduArGQN8q2aDbIJLL0+DWPZKkHmR18b3ofriyZZDwq4wx1AuzgvahP1mO1YNzS sIq5jkofdt/b/pe0knGkyiwePZhV0n/cT4MbhBGG2/+wNV09nba7lsZq89jQY5DXFX3l QLXw== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=SVc8FyUj; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.85 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR05-AM6-obe.outbound.protection.outlook.com (mail-am6eur05on2085.outbound.protection.outlook.com. [40.107.22.85]) by gmr-mx.google.com with ESMTPS id f1-20020a056512360100b004b069b33a43si1259303lfs.3.2023.01.05.00.19.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 05 Jan 2023 00:19:18 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.85 as permitted sender) client-ip=40.107.22.85; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b=SVc8FyUj; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.22.85 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dQY0eAMIV9wZDq6u8MW1miVUm23pBtmwnQbazO9HUoG2IVU9lGeTrZJ5ZLcmIUZ/HUG6bAjMfpV/MUi/8fMg1/H45wAv+uew09dCmoWLidhpXQEO8opWX6nDP5rVEgx90HKNwYn86TqwJu/r6fW8K96fULNX/tDtAHP5xPMiq+yCbCQLXsdbLSsFmeL1t//+/4t4pTLALVfaQRoZuF4o+blIXfnjCw3pbF6QHET3B/YuNmhXWTdh7YzACURzy8oyvgHEYAE0y3nHfEvP3r986mdE9y/Be/imyBVWW1jhZXKaV90GjKwhrU9NLFsAyedcQiLbHvaL+i9Zhu9MeHAIpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0p1olQpyP5dd91Eel7i6RNQkH2EzXpbabk5TefYpWLo=; b=ZZ1J5OAD9mFL4afzvdzkh2u2CkzO6qcDAD+aUSdG5EcvtmheaPApYMv/qNSudMKLKr80IBWwmI2j6LDuTwJzcBr1T86W/eXBUfonn0s5BSKgp9U9lJI6dCaDulL6asKngpKL86q1xLA6GlNDkMNB4/ZXjMKYn3xT4kv5BKUtk4MBfxtnBINA2nQsR2fCxlOr6mnOpOoHWQrXT8qqP9MXsCo3BzXW20Uk72RfLIdZe08G5E0P9oBc+N4eH7g1Fk2q/H0Ahj3w97BoO9v4dhcbzMltPvWxQazfQbnwQvW0P5wXnxWUo2+vc0VQ63RYWRz0KBb6Kqj+4DM0TZ8EYJp/Ig== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0p1olQpyP5dd91Eel7i6RNQkH2EzXpbabk5TefYpWLo=; b=SVc8FyUjlZAarcpAANUZSFaFPFLGPm/ewLGgHeb2gVkO4WoLePx6yV6H27IA+CL0avrY1GN+X2m5vt+9ncWBGEBmbkpEmIAjg58Lv43nfwAHrd6pTpuqa9jFWPjmew5qEn/IlylLTBjjwcs6IAZbQiDeMdXP3wxOJibggVfn9OHKVFMHTqOsMAKx9TtQoJYh3Hr4E+THzFyv0sDbrFtGr0Nnscopaiybpv6Soguu+8dHfVxOXfqmzXvS+KOYsbaWuASuphtPVg7ukI5WmQk3qiwZbDyPdNxtnsBiHwsh8nLvuS3J/ioPHsP2Tom8/t1Ms4Y9czHjAAr3G0bS7eofQw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by DB9PR10MB5258.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:10:33c::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5944.19; Thu, 5 Jan 2023 08:19:16 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79%3]) with mapi id 15.20.5944.019; Thu, 5 Jan 2023 08:19:16 +0000 Date: Thu, 5 Jan 2023 09:19:04 +0100 From: Henning Schild To: venkata.pyla@toshiba-tsip.com Cc: isar-users@googlegroups.com, amikan@ilbers.de, jan.kiszka@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: Re: [PATCH v2 1/1] image.bbclass: fix non-reproducible file time-stamps inside rootfs Message-ID: <20230105091904.530199bb@md1za8fc.ad001.siemens.net> In-Reply-To: <20230105061857.14993-2-venkata.pyla@toshiba-tsip.com> References: <20230105061857.14993-2-venkata.pyla@toshiba-tsip.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH0PR04CA0089.namprd04.prod.outlook.com (2603:10b6:610:74::34) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|DB9PR10MB5258:EE_ X-MS-Office365-Filtering-Correlation-Id: ca0b4d57-0a7e-4757-9e75-08daeef5893f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CDtzdtB+M521GMPReQWVr7DNKCEripPs8R+m7OgTouZb/r9GV/mVR9GOUMZaMNqh6lABL8c9bOxnQE9GG2Gm+9Q88bNikDbgLbCvW0qibDaChpBUK135PfGGqPJ6/+DOAlKwJMGCipIbXQA4owUw9GHcQpCwW9v2Ymb/3GvKXKlIIPVJ8QWIpzcKFuGPWyDrlcNlDgylUZLvzZq/VS1+8uVbtawd1fRbU4qP2ugsERxCcjcg7vmP8OcUun7CwnqYppKYwJj6+stDkx6/FrmJnjn3Rqr7YMuM6kLDAK+XtbngB+2oMIqyn99iM/KveeGPMKWysAqk2ulRgfwyaB1VG4t/a8zZB1u5NZcxu/RS5Pq+SrVi3NGCVZ3a+GhZYMg9Zq9ov/+zzFWn58jR6jt62pEmYS4aRDTC9V33WQBMKVl3y/+1RNOgtmhZgh8NhTDSY2TF5Z8Uw6kZkyNdc+Jbqg5bWCsCZvRVEYz9+gNbi1dfb7xexyLSRgZMZ5iJyBQAFsNvSgyvI8T0TMVmISxpj/DeYQqIvnLXjmM2+iJXeC2FDpIZMgfIvK1R6c7XiKXY8nINUsqFapPGMRn3qOyJ9XLMSkb30Y88vfowbgcMEqaTwbi/T2wLpBdxowXKm/bGSrsfn3YglHIgP2iSjgUpjBxXTFrwu9T8wq7pliCLbWTtEACrxFi47Y8v25ZbNr3k0F+X2WilWJ747EOHsPPCmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230022)(4636009)(366004)(39860400002)(346002)(376002)(136003)(396003)(451199015)(44832011)(8936002)(5660300002)(2906002)(83380400001)(41300700001)(86362001)(38100700002)(82960400001)(186003)(6916009)(66556008)(9686003)(6512007)(6666004)(316002)(478600001)(8676002)(6506007)(6486002)(66476007)(966005)(4326008)(66946007)(1076003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LK+2tafDmndhVEARUqhk7lj2oQT4YSbHwqjQyXJ/EWWgJeLbBalTQ+GEF3Q2?= =?us-ascii?Q?JmVqzdC2FFzGuLrkDC4ZLca3uNaOqGZ5EYb0B6YvMEM66DiL3QJ2OBIO3zJl?= =?us-ascii?Q?uB7mYJ1B5+lpBoZe1oFU5rn7ePXuyxLzrayIUF7y+yhhxlKTq4i1iRKdqrCO?= =?us-ascii?Q?/vRabeWS3bpC3sttSlxOtKK2D4XdblVtGOpMAMzg5GD+8v7yTyj3gja3AYMt?= =?us-ascii?Q?dldEh6d+hDOM2LByws03YaebgCP6BradJxd4lo1m3i+GWZMNpfXxU2oGwn2l?= =?us-ascii?Q?OcbYOgIKVLIgDQsWqeymPmZgsR8O8rUmDlyqXHKG4dZcVM2MRpo7qRyzuaW0?= =?us-ascii?Q?fRRS73tiuPc/Q0z9DBKAkpn9qLoX8KxMlyDmh42VlJ6nyKfCPxDL6JfnodrU?= =?us-ascii?Q?GRgrh0bAanMLfAmOyifPLBR1+amyCEwIPDx347m2FPdS+mxJXa7ir7lBq8Cy?= =?us-ascii?Q?oK3/pl/0VNmkQrrUqZU7Pvy+zff9Xm3ZzUb279jfoqpT85gYrwjeoUO8e4g6?= =?us-ascii?Q?VdqhGkC53yP4CdVfar8H8fzjVQir+HANCScnod+r/wx/KY6sqTBsT2+qFz3j?= =?us-ascii?Q?0VIxOX/E8gGF8q2U1VLYWysLVxTruCJQVZo5voDtA6PQdbYGXgyXiVpxk83p?= =?us-ascii?Q?jlgKr1E68WQKqATGGWMJCVeMX39AF3khWjVrdbh7XdJxA65gH5NFks5DGbVm?= =?us-ascii?Q?/nex2OxvQ5076XGNIz5n9TPo3a4HGNsHuuyrn7SXBDiCF4vaQ75PAP57jYhL?= =?us-ascii?Q?coZ1GmCy5PvYHcT5SDsBFAExwsloK558vbgt38/+aNKg5ei6D2ApJJUxcj9U?= =?us-ascii?Q?grQFm9GYb8xEIbe2rmibt/eR46se9Vsgp5skh8cJCuZ2V5D0ybbAe4ZSkDNj?= =?us-ascii?Q?igfczFGO6HcqJCNjupazBYmP/j7kx7SUP4XY21xdIuNdevCyMlKLeHsgcOKq?= =?us-ascii?Q?h8r29hoQiOdu+mXXrNmw8FFEQ650mSWhKXuwpXiTxJuFv9mT0FEfViTP6IIB?= =?us-ascii?Q?YjD8m/ozouc0mbjGb5KH1FioXHEM2+cw/sK+c1WSs/dhIyPNu1lOdlGA/iEV?= =?us-ascii?Q?OfGLMe4SD9v3mPZR6wkirY7QfUg3Q5yoU+h1siirx7JJ/Xvq5Sznl9qE5PJg?= =?us-ascii?Q?deqVGNhteLIseKjYAeWeBkH5NQXkyrrVehdcinNVscAjbAE6CWwlbIB9xS0X?= =?us-ascii?Q?btTi+tOph3+jE5X7hMvBIac4uVylMwy/Xjqe5YjLptmX+lust3uUWDZ5zEye?= =?us-ascii?Q?kaSRIpR8eFkhWqXP+JKioZJIndBMK5UivRIB4UWMiBLrsmM0780mCf6iK4z+?= =?us-ascii?Q?QHvk2ZfIvMe0LAqcGF1F1M4ETV+IKHPUdWMkAgCpLXvbcWyzsee5hHGoaUGd?= =?us-ascii?Q?0dxHQLINydb+j5MJuh10jxgJUFGbsCH02vruzgYFDjKaGR2AUOC608H6Kd3X?= =?us-ascii?Q?tragtxYm4yBRWFuvbW5C9/6Vw1+uflxp1FfcyAcVRgnGl9/EEkJhhyKWzMEP?= =?us-ascii?Q?fkE23vLxIV6EX/hdknnNChjVn87HoJoHdGJ2y+nTOll2jp7/E0hVaq9ijX9p?= =?us-ascii?Q?20nF04ziUYfPxm2iSHML4ZQVNX4VKAY7+8KP1dlSXClXIp4qtypC6ry02Ilg?= =?us-ascii?Q?nCaaqA5w+dTEZeZRxoLXyEJ8pQ+WIi2ATn5KEolurwci6beXHBJ37frHhFRu?= =?us-ascii?Q?GQz13w=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: ca0b4d57-0a7e-4757-9e75-08daeef5893f X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2023 08:19:16.6360 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5zS/VnhSAm8/x7hdmAX3tV9uz+nodz0abYgvpVTRmrsCCR9Y+2B2F2mADKTW4AaP4bHGY0gxQ3DVRnlEkqkXid/CfrWIp9/xYf/s1q5NzoU= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR10MB5258 X-TUID: VuguKjtTXnf+ Am Thu, 5 Jan 2023 11:48:57 +0530 schrieb venkata.pyla@toshiba-tsip.com: > From: venkata pyla > > As part of reproducible-build work, the rootfs images generated on > same source should be identical between two builds. > > In this commit it tries to solve one of the non-reproducible problem > i.e. the rootfs file time-stamps generated during build time are not > reproducible, it uses one of the solution provided in the debian > live-build image project (refer [1]), it fixes by finding all the > files/folders that are gernerated newly and set the time-stamp > provided by `SOURCE_DATE_EPOCH` environment variable. > > [1] https://salsa.debian.org/live-team/live-build/-/merge_requests/218 > > Signed-off-by: venkata pyla > --- > meta-isar/conf/local.conf.sample | 10 ++++++++++ > meta/classes/image.bbclass | 9 +++++++++ > 2 files changed, 19 insertions(+) > > diff --git a/meta-isar/conf/local.conf.sample > b/meta-isar/conf/local.conf.sample index 57d0620..3c4a473 100644 > --- a/meta-isar/conf/local.conf.sample > +++ b/meta-isar/conf/local.conf.sample > @@ -255,3 +255,13 @@ USER_isar[flags] += "clear-text-password" > #CCACHE_TOP_DIR ?= "${TMPDIR}/ccache" > # Enable ccache debug mode > #CCACHE_DEBUG = "1" > + > +# Uncommnet and add value to it to build images reproducibly > +# > +# The value for `SOURCE_DATE_EPOCH` should be latest source change > time in +# seconds since the Epoch. > +# Git repository users can use value from 'git log -1 --pretty=%ct' > +# Non git repository users can use value from 'stat -c%Y ChangeLog' > +# To know more details about this variable and how to set the value > refer below +# https://reproducible-builds.org/docs/source-date-epoch/ > +#SOURCE_DATE_EPOCH = ${@bb.process.run(git log ...)} would be nice here. So once uncommented it will keep moving as people commit. > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 813e1f3..38a9adf 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -430,6 +430,15 @@ do_rootfs_finalize() { > "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > > rm -f "${ROOTFSDIR}/etc/apt/sources-list" > + > + # Set same time-stamps to the newly generated file/folders > in the > + # rootfs image for the purpose of reproducible builds. > + test ! -z "${SOURCE_DATE_EPOCH}" && \ > + find ${ROOTFSDIR} -newermt \ > + "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d > %H:%M:%S')" \ > + -printf "%y %p\n" \ > + -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' > > ${DEPLOY_DIR_IMAGE}/files.modified_timestamps + > EOSUDO I would suggest to at least display a bbwarn if "wc -l" of that file exceeds some number ... say 50. I guess if SOURCE_DATE_EPOCH was too old, say 01.01.1990 the whole filesystem would be touched which might indicate a problem. Not sure what a good number would be. We could also check for certain files to _not_ be in there for sure. I might give that patch a try and see for myself what a too old value would do. But right now i will keep going with the expectation that it would "touch all files without big warning" and the thing might still boot but the broken metadata could cause any kind of problems in applications that can get confused by that big change. Henning > } > addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess