From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7184069459095584768 X-Received: by 2002:a2e:5405:0:b0:27f:b2cf:710 with SMTP id i5-20020a2e5405000000b0027fb2cf0710mr1304937ljb.263.1672677870206; Mon, 02 Jan 2023 08:44:30 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:ac2:4e8a:0:b0:4c8:8384:83f3 with SMTP id o10-20020ac24e8a000000b004c8838483f3ls109826lfr.3.-pod-prod-gmail; Mon, 02 Jan 2023 08:44:29 -0800 (PST) X-Google-Smtp-Source: AMrXdXsCbfdNO+1ZHg2fA3fRrs0MdvYJhvAQz3mlF+rmgZC5nXXCNemc78lk7Bg7mxQMJ+PpqImw X-Received: by 2002:a05:6512:3d93:b0:4b5:a207:8d70 with SMTP id k19-20020a0565123d9300b004b5a2078d70mr14075106lfv.5.1672677869157; Mon, 02 Jan 2023 08:44:29 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1672677869; cv=pass; d=google.com; s=arc-20160816; b=0ZJCqee1MEqpr8BtE0ZyMTR2+xY/0RwodKwFX/KztTOqdI5eRGIHiRfXH+DQhLt9b2 btbP1YO4bB4MaVx6r9X1jV15VV5KjgA7S/1YGvOeNRhlncDE+VOLYdkeGmo5DSAXdHoC 2zadlcNl7/IRFKsYHGcPYzCv18mrDRf45GMdLWOwBKqJEtj4/G3KvqKSzIgx9fpD6W9B RmGOG0WZPlKsHd7bsmHvjL3UFh7PTmEhJZfdQASgNQsN99t4OjkjbbaPV0qYa8FAajoX Dv7fg7/j3xumqeMR5DdQeKrvqzkYl0o0THELy8yCzlGWaU/6UBSo8jCI7eSvNjEcftUJ 4sPg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:content-transfer-encoding:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature; bh=6EgDm+ISySO1lnEw5GGnSTQVvRDjpnjZkDNwA+f/Gck=; b=C96KB7N8UbS6oyKRk89Y0bp1geJFk1PJeFXaSZJWREpjKJ5DNJ/QSFzHV/E7OmUJK4 1B0JBcWUo5gkJeshA5Nh716+ikrh27p8bZSNa6sqm4NxLEmvIXLUhJS+fjQGoMG4RXQH KYQM3MB1PJ8+i4e5gkfhSfuEFgev3LDbp4bYVkrqE0IkeCu6oysRbDNOHMzSyv0PVmNd kGGexcGaYyzUmZ8kdn55XXtiQv74SvJexOEelUGI8ITw30CIgDa0ve6EuORNsP5foOpW p98/gbEK3++fQbN3I7S5ponyJdRGBcLKzPJkp7z63T26IdvyEB5FMSrYmHaVuLgSE0Ow QEUA== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="oUOf/f15"; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.104.48 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from EUR03-DBA-obe.outbound.protection.outlook.com (mail-dbaeur03on2048.outbound.protection.outlook.com. [40.107.104.48]) by gmr-mx.google.com with ESMTPS id s4-20020a056512202400b004abdb5d1128si998149lfs.2.2023.01.02.08.44.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Jan 2023 08:44:29 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 40.107.104.48 as permitted sender) client-ip=40.107.104.48; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@siemens.com header.s=selector2 header.b="oUOf/f15"; arc=pass (i=1 spf=pass spfdomain=siemens.com dkim=pass dkdomain=siemens.com dmarc=pass fromdomain=siemens.com); spf=pass (google.com: domain of henning.schild@siemens.com designates 40.107.104.48 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hxucmf+ShRmkwypfcad/hY4nJlS7NbgHX4AVoBap6P+6ROwSs+9AG6dDJhuzUinqX9RRIg2DdrKzqXuPPYEfjTAa8cT1H5nEoI7pT8Y2g+yrcLDconhTvRNTPevbGPbk7Ks/l5kH+KdfOSXs8ECmC8TwOczxGMKQYp0N3LGPtghfl82MeQ/EnIVOfks3ILji7g2tG0yvNrwk+6wFcMRQLeuk7Tju5kc7bHIedtx79+rUyx4PRz4dK2zXL5upt5YDt0mYyhNaHD9p9zSWmzUo1USCX6nBddRbgiBcBe4OQ9GCcfpq/4Ga9hyRviSLWUcpjNbv3BlFazutL+AJ+jMdzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6EgDm+ISySO1lnEw5GGnSTQVvRDjpnjZkDNwA+f/Gck=; b=m3P8SDq1tBIPhSx56DivNENS29r2haCNYqaOoucdk8hCzalr1BK7j56OyEG00Cnrx23Ll/Zj7WuqYZwK+JkbxMPZlH5pWNw/p0oUDziXcq6hjOQgNM+xpcqb+41mY5fSUH/40iSLSZWnD6torj9E2fpwaCh9huyv3jJ4Xywnmn8LECciTcLKQTESY7NUe8QLxPvZPBoAxM7ZnJO5h9LaCIIGuJRVvzwGYpKd8QvlGVpcT5yOFWyjwNZD7hul5UFYIobgKqTOSREBW85EZ2KLb1NoasvOgzeeZn6AXmTQi6D1+QCqZPv847aCZmJB6EHzkgGxj8ZiPnTbshESkjIvfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=siemens.com; dmarc=pass action=none header.from=siemens.com; dkim=pass header.d=siemens.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=siemens.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6EgDm+ISySO1lnEw5GGnSTQVvRDjpnjZkDNwA+f/Gck=; b=oUOf/f152WQcg1RRVEq8yV2ZiNMmr47SbzqHnpkKXpVlpZr//FXk7Kp/JnT05gAHQcZJgPPb0TLwqhV1Wf0TQpl1U1j715IJvaRlPEiEZvw/+8jHrcgkA9YMOhqvGUABWwEAwDIhBiqnV5teI6k8oj1meNiamucb2hk/kuX2NH8DGV56Ab3MOfCWOVZIomhMQMFIqPwkROuAd6I8ba+VLgJYO4n5VhbVpY1Nq5wA4xWTmWsvXBq0qUgu9QSJkYlQPBPrPkNm43CRq+IUB7Y/iDi/J9kKiSIguRM6OGdtSI3XYbF+YrvnEgE5PFU2Zts/pfX4jrsbxs3LqIdKhCemng== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=siemens.com; Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) by PA4PR10MB5753.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:266::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5944.19; Mon, 2 Jan 2023 16:44:27 +0000 Received: from PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79]) by PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM ([fe80::bdf0:fdeb:f955:bc79%3]) with mapi id 15.20.5944.019; Mon, 2 Jan 2023 16:44:27 +0000 Date: Mon, 2 Jan 2023 17:44:18 +0100 From: Henning Schild To: venkata.pyla@toshiba-tsip.com Cc: isar-users@googlegroups.com, amikan@ilbers.de, jan.kiszka@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: Re: [PATCH] image.bbclass: fix non-reproducible file time-stamps inside rootfs image Message-ID: <20230102174418.686715cf@md1za8fc.ad001.siemens.net> In-Reply-To: <20230102145828.32763-1-venkata.pyla@toshiba-tsip.com> References: <20230102145828.32763-1-venkata.pyla@toshiba-tsip.com> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.35; x86_64-pc-linux-gnu) Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-ClientProxiedBy: CH2PR14CA0019.namprd14.prod.outlook.com (2603:10b6:610:60::29) To PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:102:269::8) Return-Path: henning.schild@siemens.com MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR10MB5780:EE_|PA4PR10MB5753:EE_ X-MS-Office365-Filtering-Correlation-Id: 957cae4c-bf7c-4f45-1a1b-08daece09cdf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sNuZcegho0KhQDyVsA6ih1D36ygbiXwKUcLh8CBvHBn/1I3fUAhygSnbe2SaayRbXCG3uLlmbOos46BYKDwRF6MinBhi9Fc2tP0G7AABWHizf+qFKvOeImJ177sk+VvjjC1MsIHvxfP4xYx/7PyIJyc4WcuXjvujPFzcJ9QyAra8S/b0Xp+JUSkrwxpwo77Z+P5ltMm+qTA7jQzVx0zSDXZjZn9j4/7o7cWY36PfEqJvTwJjZ+C94WfYPkvuWBJfwt36hd2tUwz6AkUGlCPk2LXtb7ZM6cOqHtXl1vzd1lEbWLB9fylSkIzZXLLkULh7g17IqHr1gfDz32XDbp3smKTy8ryn3wqg8k1nn6KlQMo7KMsj8NBdnVkYX+D5SRTSiWmAkVUqt63A3AvrJJhSqLO8HKMEIuK86JiPHx//lvTVRoWr/Sd44OIaWUQ4rju+DCylBDjLUggnd2Qj3GOLXJHLytReesuEjROc6HAo7LahNyQDrEVKZf9UB5dKBsrcOqLbUNAJY6S0AbVEJb7xTl+PUDor6as0vU/APuvUFwFqTSaUNagGWKtq6fHPSpH4ilAkY7HOl+wngX0nhvFtEk4cNfrAbD0PLq9M2CzD1km7gwtPadwXYCkONo2lRhaP5gGuEzr/CgZ9gsezFir8beggzehL0mmD3JXFRDdBIGK6wtGLXjKphACXjRGpNlyFSy/YvIFxKcn28JZN4RcQz93U8TAn0ttN/vXHul0RAX4= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(13230022)(4636009)(136003)(376002)(39860400002)(396003)(366004)(346002)(451199015)(66946007)(66476007)(66556008)(6506007)(41300700001)(4326008)(8676002)(83380400001)(2906002)(86362001)(186003)(316002)(6666004)(26005)(6916009)(9686003)(44832011)(478600001)(966005)(6486002)(6512007)(82960400001)(38100700002)(1076003)(8936002)(5660300002)(22166006);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8C5TNVcWMv7cg6wxj0UpmP0N7zlE5itWBBYTvZ9fZ9CI1Wo3r/ANA8E7rd/B?= =?us-ascii?Q?4zyFcb/l5Axpzj/AYwoEvx6DJW+NKT3x2iWLF48LJuxgk5fHdSQwrNtghUh/?= =?us-ascii?Q?tvwOuev4bjfrj2oUK5oN6Vo7kxdy2ngztHKeS8LImFJXEsRrsiWxx5T96281?= =?us-ascii?Q?ecSFgFFytDeF/rt5HgCcJwu21gykITQ4sraCCFT3tkQ2vDHhKBOGHsn2qtSt?= =?us-ascii?Q?l9+39y6Enkt+DAodW/hzHHjlhn31vYzrMKrkLHbIlhP0VeZOdsB3kIEMG53a?= =?us-ascii?Q?ZWigO+OLziPLHJirTFkmqxJJfZYyR/NY/OLP535xmhE9oDs2qVbQi/m/CMfS?= =?us-ascii?Q?sdXP65h3PBJG2YcBl88qb44PvuHA27XOFCdavmr5mE5LrbOlmcRKmL8ACk3r?= =?us-ascii?Q?TFx+iag9hn5veFDDa34PXEoG61Bp8PbOr4hRMJ5JnA5s+21tAtBWQqfsGDjf?= =?us-ascii?Q?xyzhEPCu8AOE+XP3dx1VmpWcCJ0f0okvn6QTwC9SWPoGuTpyCOpzP8WMs8Gc?= =?us-ascii?Q?7ylFIA6vYR6j9G05SHUsE9/K3wbEmnbj0/4IvvBsjgdgQNj1R0eZ2Lh0KUrF?= =?us-ascii?Q?gpfGSPU28HC3Obflbs+zCz7UQaeiRh96uc0pHgyR1lmDAqw0KIkV00CoivU2?= =?us-ascii?Q?PP/CA4Q6L1X9kiKzPDXYVNFINFl42H2iHdRi1aCnya7LfvsJJIry4Krd/gtw?= =?us-ascii?Q?BPKS5xSzJWFVbn+2xrTg4im2JWHAzZrRfPIh2J+KkpWfr7mdL+CEdU9QIVrV?= =?us-ascii?Q?AwLdOnvFNClI/dfj+abHapLzN79YYMUCrGoGtZYBlPVMEZo0jM2d3fmGGZ0U?= =?us-ascii?Q?pT75bwQYr2bbP+pQBR6i6pWa1K80Ngw+lvVQRQkPLcS7fxHjhcBUbdilCJrt?= =?us-ascii?Q?BNmRj4NjH2yucPWUXCmL3xW2FLwQDO3EcY7iv8lA8ph/HxYDC2wMpK6lG76j?= =?us-ascii?Q?zO8pKOSzGFXrKJP9PhT09msgHYrKTFjCx1xsm65SPahKwuKikwOpG4Tj8TLB?= =?us-ascii?Q?twA0XyVPhNkvRESHO9YZabGLkaKTTxk14gG3GeOpL0cFqlT6bgJP3YcW/Da3?= =?us-ascii?Q?lyAzdUMc05ER0S6mqq1IH3VUAwXWgq3N3obmeb8GF6NbaBcQFgfvrrv1jy+7?= =?us-ascii?Q?QRjfx4JAsknx4FitMlNtiwyaLY0ocizJmOD9cKY/xStLbZ57hOkPeIFn3vtv?= =?us-ascii?Q?uH2aW1NNfWJexvn6yx8NbDcio2cWu1xOurglz4CDZKs92Y2EFL4+J7S/ns+N?= =?us-ascii?Q?TD0+SN0Ri5aRLOjegCZ/4fzv6CkizG8GpgqkARqluNpCbVJ0uWaj8GzJKeJ6?= =?us-ascii?Q?bcybpTdhB8sSWue7OaE2YITCFj/15+FCd5OxGWMoqm78yCldgswVLhOO7zbz?= =?us-ascii?Q?R7PMEOVjqGeKxHMvru4BmHqZqkxasCVOMNWlpvr5wiujmOj5puEJGbYlIuwy?= =?us-ascii?Q?OwbkX18PgsO7J7WCOnhpzHwqEavxeGi0Gxkwl+hSxhEUAKBPX49QhS0Yj1NG?= =?us-ascii?Q?UdLbUxB9NKx7byCLP+E6saXSBGtmHJBP4Nc7KxtXn6rfEtLf7IePRnGBQNnj?= =?us-ascii?Q?wd15EgeUqHJ7GtsoYx3D8Z5rkoy5VPwxFdA0oQJi47++xwtUc509DpUjfgi1?= =?us-ascii?Q?YA=3D=3D?= X-OriginatorOrg: siemens.com X-MS-Exchange-CrossTenant-Network-Message-Id: 957cae4c-bf7c-4f45-1a1b-08daece09cdf X-MS-Exchange-CrossTenant-AuthSource: PA4PR10MB5780.EURPRD10.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Jan 2023 16:44:27.5667 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 38ae3bcd-9579-4fd4-adda-b42e1495d55a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UVyV6sUClQ3N9P4Lw+Pd+xMJLlUmt4EeGspW1gFrtZzAlLz9L48Z3FWpq1B0/dk2Gpg9uf34qgrm1YECLu1tnqKcErhOMx6zXGN/xiBV6/U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR10MB5753 X-TUID: f2mTWxsUo8BB Am Mon, 2 Jan 2023 20:28:28 +0530 schrieb venkata.pyla@toshiba-tsip.com: > From: venkata pyla > > As part of reproducible-build work, the rootfs images generated on > same source should be identical between two builds. > > In this commit it tries to solve one of the non-reproducible problem > i.e. the rootfs file time-stamps generated during build time are not > reproducible, it uses one of the solution provided in the debian > live-build image project (refer [1]), it fixes by finding all the > files/folders that are gernerated newly and set the time-stamp > provided by `SOURCE_DATE_EPOCH` environment variable. > > [1] https://salsa.debian.org/live-team/live-build/-/merge_requests/218 > > Signed-off-by: venkata pyla > --- > meta/classes/image.bbclass | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 813e1f3..f592a12 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -430,6 +430,15 @@ do_rootfs_finalize() { > "${ROOTFSDIR}/etc/apt/sources.list.d/bootstrap.list" > > rm -f "${ROOTFSDIR}/etc/apt/sources-list" > + > + # Set same time-stamps to the newly generated file/folders > in the > + # rootfs image for the purpose of reproducible builds. > + test ! -z "${SOURCE_DATE_EPOCH}" && \ > + find ${ROOTFSDIR} -newermt \ > + "$(date -d@${SOURCE_DATE_EPOCH} '+%Y-%m-%d > %H:%M:%S')" \ > + -printf "%y %p\n" \ > + -exec touch '{}' -h -d@${SOURCE_DATE_EPOCH} ';' > + This looks like i have seen it before. For me that is _way_ too generic and something that is not a package touches files all over the place. If some package now wants to intentionally bring a file that is from a far away future? Which files are we talking about here? It can basically only be metadata and other little places where we violate our "everything comes from a package" rule. Has this ever been tested against a complex layer, has any of the repro work ever looked at something bigger than the very artificial isar base image? I think a better start would be to bbwarn and only much later move to "-exec touch". We recently added CI tests for reproducible building. Would be nice to the two patches. p1 writes a test-case that goes red, p2 (this one) makes it go green Henning > EOSUDO > } > addtask rootfs_finalize before do_rootfs after do_rootfs_postprocess