From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7276045224940404736 X-Received: by 2002:ac2:5547:0:b0:500:9a45:62f with SMTP id l7-20020ac25547000000b005009a45062fmr4389371lfk.8.1694086292094; Thu, 07 Sep 2023 04:31:32 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:6512:0:b0:4f9:5662:5efb with SMTP id z18-20020a196512000000b004f956625efbls816588lfb.0.-pod-prod-01-eu; Thu, 07 Sep 2023 04:31:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IES2V+Ek43698WTHwJI9RaZcpbXohMXHYRmmNguoszuDwQfF94vobfGanJTnflEquoMPXg0 X-Received: by 2002:a19:430c:0:b0:500:9031:bb1b with SMTP id q12-20020a19430c000000b005009031bb1bmr4422856lfa.41.1694086289884; Thu, 07 Sep 2023 04:31:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694086289; cv=none; d=google.com; s=arc-20160816; b=wtXxFb3Wurwt8TyP54/PsvFD6Vps77QNYpibN7L79fBimpm43XrhzMU6CNxnGvwbjy 1y/PelGxRqalIeRb8yJu6RceJzOR1UIu3p1xHUVU4hPvx44xP0ynxuB7Y0Ex72mfgWfa 7i01ss4Emo/O02H1iu5LGH8EkpkXQKpo0EfzKsuOAW/y2QcgPymCogRN1rBxvaiHAi81 1MmKoYe0Rx/VVPnpvogXfi+TSY1UsTnBhbA/wbvMQoM1aBOWMpOVu0LkrBQDjj8kEGf+ 7O4l93Fa3HIpEQ51pH650wBhizbgPnzPUhK6cJEWNUFEuhVqcbUxad0pIai6ILvkI/LB D/Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:content-transfer-encoding:date:to:from :subject:message-id; bh=h6Or+pbRFhZxOT1phVyC30purmGJzWZ09BqdiUlr7JA=; fh=7tclEdh7YbwSQowgJ6LNq720O7H5HTEaqj22NJWRE2E=; b=p+O6jHlTY5Nz5oCvIm9ZDX1BH6pgEsMv0PBegOUOuK4UOxBtE07zVPn4sFwrruXN8A pUwvtqVSONfzrhN7C0LcMPYp9YyO6kb0UBJy66AiH4SDcEgz5bONcweZSasep4ro26mz WHLHHCScp3BAmG2tT1J0ENd1rRs07lZZsYiAERtttziYhvyIHDCD3yfw3qeEy6J6CWcK SZcX738lReO7LIMPyRxToK7RuDia1jH49/nbVk7YyK6IyJH0UHysx2sf5VwIflSQtyJk GiOl1eJcQoapM/EU9fpv6OKS4gTyT50a2aUzEwRT06+YRLK1m1KwixktGTU6d4pZVP2c y/NA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id j18-20020a056512345200b005008765a16fsi998636lfr.13.2023.09.07.04.31.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Thu, 07 Sep 2023 04:31:29 -0700 (PDT) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from [IPv6:::1] (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 387BVRPA018901 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 7 Sep 2023 13:31:28 +0200 Message-ID: <7f715c07ed85772dc5e2eafcb41f3ace734de9d4.camel@ilbers.de> Subject: Isar can't reuse sstate-cache build from different isar location From: Uladzimir Bely To: isar-users@googlegroups.com Date: Thu, 07 Sep 2023 14:31:31 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.48.4 (by Flathub.org) MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: bBeBIzpmWLqs I tried to use the same SSTATE_DIR for different isar locations on the same machine under the same user and found that it does not work. Quick steps to reproduce it: ## Prepare build environment with two copies of isar: ``` ~ $ mkdir -m 777 test-sstate ~ $ cd test-sstate/ ~/test-sstate $ docker run --privileged -it -v .:/build ghcr.io/siemens/kas/kas-isar:4.0 builder@ffe5274283d3:~$ cd /build builder@ffe5274283d3:/build$ git clone https://github.com/ilbers/isar.git isar-1 builder@ffe5274283d3:/build$ git clone https://github.com/ilbers/isar.git isar-2 ``` ## Build isar-1: ``` builder@ffe5274283d3:/build$ cd /build/isar-1 builder@ffe5274283d3:/build/isar-1$ . isar-init-build-env=20 builder@ffe5274283d3:/build/isar-1/build$ echo 'SSTATE_DIR=3D"/build/sstate-cache"' >> conf/local.conf=20 builder@ffe5274283d3:/build/isar-1/build$ echo 'DL_DIR=3D"/build/downloads"' >> conf/local.conf builder@ffe5274283d3:/build/isar-1/build$ time bitbake mc:qemuamd64- bullseye:isar-image-base real 4m38.759s user 0m4.248s sys 0m1.068s builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/downloads 229M /build/downloads builder@ffe5274283d3:/build/isar-1/build$ du -sh /build/sstate-cache 1.1G /build/sstate-cache ``` ## Build isar-2: ``` builder@ffe5274283d3:/build$ cd /build/isar-2 builder@ffe5274283d3:/build/isar-2$ . isar-init-build-env=20 builder@ffe5274283d3:/build/isar-2/build$ echo 'SSTATE_DIR=3D"/build/sstate-cache"' >> conf/local.conf=20 builder@ffe5274283d3:/build/isar-2/build$ echo 'DL_DIR=3D"/build/downloads"' >> conf/local.conf=20 builder@ffe5274283d3:/build/isar-2/build$ time bitbake mc:qemuamd64- bullseye:isar-image-base real 4m32.314s user 0m3.975s sys 0m0.898s builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/downloads 229M /build/downloads builder@ffe5274283d3:/build/isar-2/build$ du -sh /build/sstate-cache/ 2.2G /build/sstate-cache/ ``` Second build took almost the same time as first one. Sstate cache was not reused, second build just added new files to the cache. I tried similar approach with Yocto, and it works in it. After debugging, I have concluded, that this issue comes from "bootstrap" task that is not sstate-cache compatible. It's caused by our manipulations with SRC_URI we do in it, so that it's different for both builds. E.g.: SRC_URI=3D"file://locale file://chroot-setup.sh file:///build/isar- 1/meta/conf/distro/debian-bullseye.list" SRC_URI=3D"file://locale file://chroot-setup.sh file:///build/isar- 2/meta/conf/distro/debian-bullseye.list" SRC_URI is a variable that is tracked in sstate-cache. Since the last part (that we add "on fly" with get_aptsources_list() function) is different for both builds, sstate-cache can't be reused for "isar- bootstrap-" and for all other tasks since they depend on those ones. Any ideas how to properly deal with this? By the way, kas users are not affected, since fixed paths are used (e.g., "/work/isar", "/build") under the container.