From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7171498873291014144 X-Received: by 2002:a05:6a00:410b:b0:56d:91d1:a903 with SMTP id bu11-20020a056a00410b00b0056d91d1a903mr38894787pfb.61.1669744698936; Tue, 29 Nov 2022 09:58:18 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:903:40c3:b0:189:33cd:635 with SMTP id t3-20020a17090340c300b0018933cd0635ls11256887pld.5.-pod-prod-gmail; Tue, 29 Nov 2022 09:58:17 -0800 (PST) X-Google-Smtp-Source: AA0mqf73kB513YVPtmDMsi4tkWIpyp5r1SRa9hD1c1xyPX2STcVS0Ybg67UFEUPTCSi8k/snPUIg X-Received: by 2002:a17:903:555:b0:189:959a:84cc with SMTP id jo21-20020a170903055500b00189959a84ccmr7023079plb.41.1669744697809; Tue, 29 Nov 2022 09:58:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1669744697; cv=none; d=google.com; s=arc-20160816; b=b9TaUSibRBcP9liOQm9fQL1IiQdR1cNd7ccDiIh7Yr2QXmZikdtSms1Oakmi75PmCc u5ASKuOK+o4PvkrOTZd2muNaVNmifGE/cUQnnd+ZHc1uhBdmL4Js6JzDMAe25O80TpSm /peKvsEs2Vbf/pqnC5fbyCwiASRkl/2jccVoqr2Rl56cZsz03mtguoL1Lhf3ge8bZu/n E5FU1TQpU2eAf2VnYeFUUiHxG6AQeH0Q1HUZFiyiJ+dkxDOoHrDf7sZURTgAhI7jUHS5 TSz+BpSEyfNHaAEKkRJqXvkzVfBZxJemgvYuPF1b6iBU+cUQP+VK6gHIkXIQo8CrSfmn JW6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from; bh=Ja29AW6VLupyLjccoR4F4JC781mPJz17iQPpeauah0s=; b=aWPcuq2nsOCw/rlY5TuVpFo+dfDEoKBttYgx+ohZj3C5JamOZ1k3lhRYu7u9PqJvyA yIA2WdnWh7E9I7yrODo2ietRkEwNNlRPseh6fbD+p4sBvzrTChDXMs7hUqzpcyfotvNr 2b8SRtF+fe/HT85suLvgBX3r2i7zUyFAQu60pvjEWvSLVEA+EZm+GGbXoWei7ogEA7Hf rFarD460ngYoPo/s/zH6zZwj1yCiX3wYJPw2LStu/RbgTsHygCCH9fw5b5M/PyN2plMa W14QrELylyZdO6oPCI0UrzydOMkMw/qaoJmNKGlMlK4bi7jgD5A/h2D9nDMUNNxa4xnR Ck5Q== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id i1-20020a625401000000b0056d189fd322si670359pfb.2.2022.11.29.09.58.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 29 Nov 2022 09:58:17 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 2ATHwEbP003006 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 29 Nov 2022 18:58:14 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [RFC PATCH 0/2] check Isar image for reproducability Date: Tue, 29 Nov 2022 18:58:12 +0100 Message-Id: <20221129175814.16355-1-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: wbg04LT2/PiG This is an reimplementation of the idea proposed in https://groups.google.com/g/isar-users/c/4ZIuKCOQzVc/m/sPUafWDgAgAJ that uses avocado test instead of custom script. The test is placed under KFAIL while images are even not close to be considered reproducable for now. Some things to improve the patchset: - Check not only `tar.gz` images, but target rootfs directories, so it should save time and allow supporting various multiconfigs. - Not only generate diff statistics, but also analyze "total" number of differencies we see. - ... Uladzimir Bely (2): Install diffoscope binary if not installed testsuite: check Isar image for reproducability. scripts/ci_build.sh | 5 +++++ testsuite/cibase.py | 44 ++++++++++++++++++++++++++++++++++++++++++++ testsuite/citest.py | 18 ++++++++++++++++++ 3 files changed, 67 insertions(+) -- 2.20.1