From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7372243272758984704 X-Forwarded-Encrypted: i=2; AJvYcCUI7M8BzggxSdpWq2iHePSo1DykDRxX+TfqYNOseCJ38QcgbfX6aJn3hRv7XlBD52D3Zw3ru3ZzYIV6rnGwWE3DUYoDqgMYtevvL7U= X-Received: by 2002:a2e:a782:0:b0:2e1:d44b:db9c with SMTP id 38308e7fff4ca-2e95af3cdb6mr14490831fa.0.1716551722903; Fri, 24 May 2024 04:55:22 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:81ce:0:b0:2e2:f76c:3303 with SMTP id 38308e7fff4ca-2e9599e9b57ls3525251fa.1.-pod-prod-06-eu; Fri, 24 May 2024 04:55:20 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCXKnj4T6elP87HLELnJLxOVoMaMYJVZOczlDHecOfKjJL6CWSkrFQ3hRFbIjg+F+RGEzVrZqM5SsnOVHazxT0fUoxCrS9YWqbPrsFo= X-Google-Smtp-Source: AGHT+IFvx/XMcCOoHFH/YOb2f99I6G7XyA15Kb+v8uiv9nfteqQugUqhLaWPU5rIGs+eVOgrN8XY X-Received: by 2002:a2e:998b:0:b0:2e1:18d:5b4f with SMTP id 38308e7fff4ca-2e95b2823e1mr12404191fa.42.1716551720396; Fri, 24 May 2024 04:55:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1716551720; cv=none; d=google.com; s=arc-20160816; b=XVXlmneKcFSmthflmbdhjevKijPf+gcnwCdLwb65sEpZ3P/VH3wxhRPUksDqGmvMfk WtKtspyRM9iZvAelCrjjBqDbi6FqNHOrGxmpvWyM4W9t9U8tfw046QYI1aEEMUXfO/8r 3Ts+HdycYyarlCUq+QwcbnG0Qo6z3xVus4SdVoNxbD0EQ/81cgKhxfCML1PoOEA/J7Ke FN3xF9LTH5UAdeQg4k6HZ7J1AsgI2UUIpceaOFhEFhSt4LvTl5H1iIeHzjVg+8zj6lUm BeA7D2tHPSWDTkqpp7cnINbS3exnJILrwaS7iY59xcHVf06daBvO4C57PpO6SEMP6SoC yc1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:from:content-language :references:to:subject:user-agent:mime-version:date:message-id; bh=Kwy6Eb9NT0+lXn2XhLe3kEKqUyPoguUy62N2rmAJ18E=; fh=aMI4U0HWKQoMRUG3SXT/a2Do/KTGKERlucnDhWPl9ok=; b=j2s1+hEMF7UU1QkFS8bQ8m6/8bM93GJdq2mpkvcAgeLdPXDtIQIOx/cSetdB5TdsGT 6knnKtvc9s3L0LjmQnpQucww5qSMW3pjm2PDi82T55vGnJHP4K+uZ64lxrzjl0IoBp2U NCq+UNJPuR6ldW4nRvq/s3iUTGr8w3eN2EKLNnKODbHoigQOS2SvJp7BIwHbXVg3hVlE M8tVhk16Ab3ixHiXtq48cGjzYavgud8l1G5cu3nWliEmeWM8G8LPdsRYOL87+kiVudRu nRDjFwzPaNWpWjhtvm7VtZIl/14btyzgEGev1dpGJ0ZuuduWGDB2vkSr2mLEVUQ84RG9 DKHQ==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 38308e7fff4ca-2e95bb5fbc5si292781fa.0.2024.05.24.04.55.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 24 May 2024 04:55:20 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [127.0.0.1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 44OBtH0h019250 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 24 May 2024 13:55:18 +0200 Message-ID: <7885df66-4fc2-49a0-9fb0-8377846cc911@ilbers.de> Date: Fri, 24 May 2024 14:55:17 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: sbuild race with sporadic build breakages To: Jan Kiszka , isar-users , Uladzimir Bely , Baurzhan Ismagulov References: <187215a1-ff1b-48b4-979f-9b49689a78e7@siemens.com> <58248c54-823a-434d-a54b-375b8e801884@siemens.com> <92b0d767-34c8-47f4-ba13-8f088674b18a@ilbers.de> Content-Language: en-US, ru-RU From: Anton Mikanovich In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: cvrbkuKz/JKD 24/05/2024 13:42, Jan Kiszka wrote: > Why speaks against this? > > diff --git a/meta/classes/sbuild.bbclass b/meta/classes/sbuild.bbclass > index f1193c20..3f27a2ef 100644 > --- a/meta/classes/sbuild.bbclass > +++ b/meta/classes/sbuild.bbclass > @@ -7,7 +7,7 @@ SCHROOT_MOUNTS ?= "" > > inherit crossvars > > -SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${@os.getpid()}" > +SBUILD_CHROOT ?= "${DEBDISTRONAME}-${SCHROOT_USER}-${ISAR_BUILD_UUID}-${BB_TASKHASH}" > > SBUILD_CONF_DIR ?= "${SCHROOT_CONF}/${SBUILD_CHROOT}" > SCHROOT_CONF_FILE ?= "${SCHROOT_CONF}/chroot.d/${SBUILD_CHROOT}" > > > We definitely need to get rid of that getpid. > > Jan > Ok, I can prepare a patch and we can try. The only point we shouldn't loose is task-taskhash mapping, but hope it will be ok with the existing logging of schroot session during its creation. Here is one of the logs we've caught on our CI. The first log output of session ID (with correct PID) is done by Isar. The second one (also correct) is done by sbuild process later. And the third (incorrect) log output is done also by the same sbuild process when the issue happened. Log: ERROR: Logfile of failure stored in: /workspace/build-auto/isar_master/1364/build/tmp/work/debian-bookworm-mipsel/hello-isar/0.5-b48d156-r0/temp/log.do_dpkg_build.3652871 ERROR: Task (mc:qemumipsel-bookworm:/workspace/build-auto/isar_master/1364/meta-isar/recipes-app/hello-isar/hello-isar.bb:do_dpkg_build) failed with exit code '1' | DEBUG: Executing python function sstate_task_prefunc | DEBUG: Python function sstate_task_prefunc finished | DEBUG: Executing python function do_dpkg_build | DEBUG: Executing shell function dpkg_schroot_create_configs | Creating /etc/schroot/chroot.d/isar-jenkins-6c34c167-bdc8-498f-828a-50cd88b412b0-3652871 | DEBUG: Shell function dpkg_schroot_create_configs finished | DEBUG: Executing shell function dpkg_runbuild | sbuild (Debian sbuild) 0.85.0 (04 January 2023) on qaton.m.ilbers.de | | +==============================================================================+ | | hello-isar 0.5 (mipsel)                      Wed, 15 May 2024 04:12:02 +0000 | | +==============================================================================+ | | Package: hello-isar | Version: 0.5 | Source Version: 0.5 | Distribution: isar | Machine Architecture: amd64 | Host Architecture: mipsel | Build Architecture: mipsel | Build Type: binary | | I: NOTICE: Log filtering will replace 'var/run/schroot/mount/isar-jenkins-6c34c167-bdc8-498f-828a-50cd88b412b0-3652871-9097eeed-6320-4cf9-b2b9-229b1d1fe5a0' with '<>' | | +------------------------------------------------------------------------------+ | | Chroot Setup Commands                                                        | | +------------------------------------------------------------------------------+ | | | echo "Package: *\nPin: release n=isar\nPin-Priority: 1000" > /etc/apt/preferences.d/isar-apt | -------------------------------------------------------------------------------------------- | | | I: Finished running 'echo "Package: *\nPin: release n=isar\nPin-Priority: 1000" > /etc/apt/preferences.d/isar-apt'. | | echo "APT::Get::allow-downgrades 1;" > /etc/apt/apt.conf.d/50isar-apt | --------------------------------------------------------------------- | | | I: Finished running 'echo "APT::Get::allow-downgrades 1;" > /etc/apt/apt.conf.d/50isar-apt'. | | rm -f /var/log/dpkg.log | ----------------------- | | | I: Finished running 'rm -f /var/log/dpkg.log'. | | mkdir -p /var/cache/apt/archives | -------------------------------- | | | I: Finished running 'mkdir -p /var/cache/apt/archives'. | | find /home/builder/hello-isar/rootfs/var/cache/apt/archives -maxdepth 1 -name '*.deb' -exec ln -t /var/cache/apt/archives/ -sf {} + | ----------------------------------------------------------------------------------------------------------------------------------- | | | I: Finished running 'find /home/builder/hello-isar/rootfs/var/cache/apt/archives -maxdepth 1 -name '*.deb' -exec ln -t /var/cache/apt/archives/ -sf {} +'. | | apt-get update -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" -o Dir::Etc::SourceParts="-" -o APT::Get::List-Cleanup="0" | -------------------------------------------------------------------------------------------------------------------------------- | | Get:1 file:/isar-apt isar InRelease | Ign:1 file:/isar-apt isar InRelease | Get:2 file:/isar-apt isar Release [5102 B] | Get:2 file:/isar-apt isar Release [5102 B] | Get:3 file:/isar-apt isar Release.gpg | Ign:3 file:/isar-apt isar Release.gpg | Get:4 file:/isar-apt isar/main Sources [3759 B] | Get:5 file:/isar-apt isar/main mipsel Packages [3187 B] | Reading package lists... | | I: Finished running 'apt-get update -o Dir::Etc::SourceList="sources.list.d/isar-apt.list" -o Dir::Etc::SourceParts="-" -o APT::Get::List-Cleanup="0"'. | | Finished processing commands. | -------------------------------------------------------------------------------- | I: NOTICE: Log filtering will replace 'build/hello-isar-gmVcne/resolver-6OF4Lt' with '<>' | | +------------------------------------------------------------------------------+ | | Update chroot | | +------------------------------------------------------------------------------+ | | Reading package lists... | Building dependency tree... | Reading state information... | Calculating upgrade... | 0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded. | | +------------------------------------------------------------------------------+ | | Fetch source files                                                           | | +------------------------------------------------------------------------------+ | | | Local sources | ------------- | | /workspace/build-auto/isar_master/1364/build/tmp/work/debian-bookworm-mipsel/hello-isar/0.5-b48d156-r0/hello-isar_0.5.dsc exists in /workspace/build-auto/isar_master/1364/build/tmp/work/debian-bookworm-mipsel/hello-isar/0.5-b48d156-r0; copying to chroot | I: NOTICE: Log filtering will replace 'build/hello-isar-gmVcne/hello-isar-0.5' with '<>' | I: NOTICE: Log filtering will replace 'build/hello-isar-gmVcne' with '<>' | | +------------------------------------------------------------------------------+ | | Install package build dependencies                                           | | +------------------------------------------------------------------------------+ | | | Setup apt archive | ----------------- | | E: /etc/schroot/chroot.d/isar-jenkins-6c34c167-bdc8-498f-828a-50cd88b412b0-2897105: Failed to stat file: No such file or directory | E: Can't mkdir /<>/sbuild-build-depends-main-dummy: No such file or directory | W: Could not create build-depends dummy dir /<>/sbuild-build-depends-main-dummyNo such file or directoryE: Setting up apt archive failed