From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7163187300010033152 X-Received: by 2002:a63:fc4f:0:b0:479:3384:9cce with SMTP id r15-20020a63fc4f000000b0047933849ccemr2092848pgk.609.1671460914378; Mon, 19 Dec 2022 06:41:54 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:902:c205:b0:178:35a3:84d9 with SMTP id 5-20020a170902c20500b0017835a384d9ls10327470pll.10.-pod-prod-gmail; Mon, 19 Dec 2022 06:41:53 -0800 (PST) X-Google-Smtp-Source: AMrXdXvEj+c7jtx25Jir4o2ctrg9IOcCA+/Hwx6sqqgdVcMEaByRLxwqF+g7D8IlVp3Wcbp6AIyF X-Received: by 2002:a17:902:e149:b0:186:5f5a:5842 with SMTP id d9-20020a170902e14900b001865f5a5842mr9337435pla.11.1671460913520; Mon, 19 Dec 2022 06:41:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1671460913; cv=none; d=google.com; s=arc-20160816; b=C4+WWJ0Y3EvlPu531SRMdcLg0SfVXOHP8TUc73+VIMMl99MpNjUYISQAGp+IExezux lU1woOVYo9NEXBlO+sl5EXn5akia37n6OhZtyZltfpYXW0AtFj1+QnlY4gM8FLxfPkam 27BnhO+oo+DbUAYacoDg8Mcqcv6NKtzMKP+DgX7yaJvBTJWT67qu9zTkej1U0K6N/fec iRvMvpiSBt4h2lV6euMV7SOCwMh/aak2rbpSRfQyKpcltmmM9WozV5hhfzTAj/09vQkg GsOzpxVU7f7pgVKU17VAX5mc5PNvGcCb8M+1BfKVpI5i5tfAiG1Gf/Ik3i2c7qJKFeK/ sl+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=WRWjSyJWt6YaonYpLowz14ZNZgEgV+5MmQbXJOWSWAQ=; b=bw1XnNAFMjEZw57AX1yCcA7RF88GUObO+qO2X9Vx9MRdN0CzHE5ufHxlJO7h96ec/3 tvZP1sISRVtnGApoT6T+MU6FhclnNZ+tJk0+o94LCxBCRWOVPSYPYnuzVSKLh31DCTyE GvQs7sLzUNVqcmqmYlmezX+zJSOtIg/S7FhPB0VXvXjMZY3MJimr0/ODapV+qpCAsKPj kAc3qBt49TDKwwniLATutlIAyhVBqZH8jzsKbU7fAPPKSPe1m7AIN+QoeZfm8d9FterA z7+9HHJ5pUEYpgifh1YdTsTfUobrLUSRlhwCqLSbm8wZAtFXskPaDh2YV2MggQFH+Q53 10nQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.154 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Return-Path: Received: from mo-csw.securemx.jp (mo-csw1515.securemx.jp. [210.130.202.154]) by gmr-mx.google.com with ESMTPS id m3-20020a170902db0300b0018712ccd6e0si972221plx.2.2022.12.19.06.41.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 19 Dec 2022 06:41:53 -0800 (PST) Received-SPF: pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.154 as permitted sender) client-ip=210.130.202.154; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of venkata.pyla@toshiba-tsip.com designates 210.130.202.154 as permitted sender) smtp.mailfrom=venkata.pyla@toshiba-tsip.com Received: by mo-csw.securemx.jp (mx-mo-csw1515) id 2BJEfpee031117; Mon, 19 Dec 2022 23:41:51 +0900 X-Iguazu-Qid: 34trDbI6Ag4CdYaq0G X-Iguazu-QSIG: v=2; s=0; t=1671460911; q=34trDbI6Ag4CdYaq0G; m=gdfjbguzTtivE/53lhzGMuh8Fqoo12ARm9i9Fp2oPy4= Received: from imx2-a.toshiba.co.jp (imx2-a.toshiba.co.jp [106.186.93.35]) by relay.securemx.jp (mx-mr1513) id 2BJEfoVa031925 (version=TLSv1.2 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 19 Dec 2022 23:41:51 +0900 From: venkata.pyla@toshiba-tsip.com To: isar-users@googlegroups.com Cc: venkata pyla , jan.kiszka@siemens.com, henning.schild@siemens.com, kazuhiro3.hayashi@toshiba.co.jp, dinesh.kumar@toshiba-tsip.com Subject: [PATCH 2/2] repro-build-test.py: Test to check images are reproducible Date: Mon, 19 Dec 2022 20:11:47 +0530 X-TSB-HOP2: ON Message-Id: <20221219144147.31245-3-venkata.pyla@toshiba-tsip.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-OriginalArrivalTime: 19 Dec 2022 14:41:48.0346 (UTC) FILETIME=[063929A0:01D913B8] X-TUID: 0lstAXzkznDh From: venkata pyla This test verifies whether the images are reproducible by checking with in-depth comparision tool `diffoscope` and produces a comparision output in plain text format for checking the differences. Signed-off-by: venkata pyla --- testsuite/repro-build-test.py | 68 +++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100755 testsuite/repro-build-test.py diff --git a/testsuite/repro-build-test.py b/testsuite/repro-build-test.py new file mode 100755 index 0000000..e89becf --- /dev/null +++ b/testsuite/repro-build-test.py @@ -0,0 +1,68 @@ +#!/usr/bin/env python3 + +import glob +import os +import re +import tempfile +import time + +from cibuilder import CIBuilder, isar_root +from avocado.utils import process + +class ReproBuild(CIBuilder): + + """ + Test reproducible builds by comparing the artifacts + + :avocado: tags=repro-build + """ + def test_repro_build(self): + target = self.params.get('build_target', default='mc:qemuamd64-bullseye:isar-image-base') + source_date_epoch = self.params.get('source_date_epoch', default=self.git_last_commit_timestamp()) + self.init() + self.build_repro_image(target, source_date_epoch, 'image1.tar.gz') + self.build_repro_image(target, source_date_epoch, 'image2.tar.gz') + self.compare_repro_image('image1.tar.gz', 'image2.tar.gz') + + def git_last_commit_timestamp(self): + return process.run('git log -1 --pretty=%ct').stdout + + def get_image_path(self, target_name): + image_dir = "tmp/deploy/images" + target_params = target_name.split(':') + machine = target_params[1].split('-')[0] + distro = 'debian-' + target_params[1].split('-')[1] + image_type = target_params[2] + return f'{image_dir}/{machine}/{image_type}-{distro}-{machine}.tar.gz' + + def build_repro_image(self, target, source_date_epoch=None ,image_name='image.tar.gz'): + + if not source_date_epoch: + self.error("Reproducible build should configure with source_date_epoch time") + + # clean artifacts before build + self.clean() + + # Build + self.log.info("Started Build " + image_name) + self.configure(source_date_epoch=source_date_epoch) + self.bitbake(target) + + # copy the artifacts image name with given name + image_path = self.get_image_path(target) + self.log.info("Copy image " + image_path + " as " + image_name) + self.move_in_build_dir(image_path, image_name) + + def clean(self): + self.delete_from_build_dir('tmp') + self.delete_from_build_dir('sstate-cache') + + def compare_repro_image(self, image1, image2): + self.log.info("Compare artifacts image1: " + image1 + ", image2: " + image2) + result = process.run('diffoscope ' + '--text ' + self.build_dir + '/diffoscope-output.txt' + ' ' + self.build_dir + '/' + image1 + + ' ' + self.build_dir + '/' + image2 , + ignore_status = True) + if result.exit_status > 0: + self.fail(f'Images {image1} and {image2} are not reproducible') -- 2.20.1