RE: SSTATE cache not working for -native recipes due to unexpanded variable

["Waltl, Jan" <jan.waltl@siemens.com>]

Hello Uladzimir,

Thank you very, the patch appears to fix the issue for caching of native builds for me - now SSTATE works with them.

Also, I see some related discussions on the mailing list I will try to follow.

Kind Regards
Jan Waltl

Siemens Mobility, s.r.o.
SMO RI LCE CZ SAF 1


-----Original Message-----
From: Uladzimir Bely <ubely@ilbers.de> 
Sent: Tuesday, October 3, 2023 5:41 PM
To: Waltl, Jan (SMO RI LCE CZ SAF 1) <jan.waltl@siemens.com>; isar-users@googlegroups.com
Subject: Re: SSTATE cache not working for -native recipes due to unexpanded variable

On Tue, 2023-10-03 at 13:30 +0000, 'Waltl, Jan' via isar-users wrote:
> Hello,
> 
> I noticed that for -native recipes, SSTATE_PKGARCH variable is not 
> expanded in the .siginfo filename. On rebuild, SSTATE reports 
> mismatches leading to rebuilds even though the recipe and the 
> resulting stamps are exactly the same.
> 
> Reproduction steps:
> 1. Checkout ISAR next 216f0721265fcfe965ac48b51a0128d612d89bf4.
> 2. Run 'bitbake cowsay-native' from meta-isar layer  (I use KAS for 
> setting up the bitbake directory) 4. Look into sstate directory, I see 
> a files:
> sstate-cache/31/1a/sstate:cowsay-
> native::git:r0::10:311a86032fcbd98d2343b40447e0558b5aae410ff64f727ae5
> 6aa2952167f05e_patch.tar.zst.siginfo
> sstate-cache/3b/24/sstate:cowsay-
> native::git:r0::10:3b2452fa0b072ed1488aad9432257dff8bee324659b5f0738a
> b224d82b1de2fc_unpack.tar.zst.siginfo
> sstate-cache/d6/ab/sstate:cowsay-
> native::git:r0::10:d6abcc3328b93b67b59ef9c0b21449188f1330bc76d6ee6372
> c76e94bec87756_fetch.tar.zst.siginfo
> sstate-cache/isarnative/64/de/sstate:cowsay-native:amd64-
> isar:git:r0:${SSTATE_PKGARCH}:10:64de5a007fac75eebd427a2ea9156cf01427
> 18f68362a06b51c65559fc8045b7_dpkg_build.tar.zst.siginfo'
> sstate-cache/isarnative/64/de/sstate:cowsay-native:amd64-
> isar:git:r0::10:64de5a007fac75eebd427a2ea9156cf0142718f68362a06b51c65
> 559fc8045b7_dpkg_build.tar.zst
> sstate-cache/isarnative/aa/07/sstate:cowsay-native:amd64-
> isar:git:r0:${SSTATE_PKGARCH}:10:aa07cf2e6ff17f7004e6dc2b4e7a9a6123b1
> 27668c27d4c3cb29d0f7dc65ac5f_deploy_deb.tar.zst.siginfo'
> sstate-cache/isarnative/c4/db/sstate:cowsay-native:amd64-
> isar:git:r0:${SSTATE_PKGARCH}:10:c4dbfbe1c1696f2b690f15cbd179a5795fe4
> 8f05a30a76d20783c98786af6d4a_adjust_git.tar.zst.siginfo'
> sstate-cache/isarnative/d0/0c/sstate:cowsay-native:amd64-
> isar:git:r0:${SSTATE_PKGARCH}:10:d00c69fa102b131bc09fb27aa8259fcb447d
> be5b51d5f6723d1e12ff43f92fee_transform_template.tar.zst.siginfo'
> sstate-cache/isarnative/e7/94/sstate:cowsay-native:amd64-
> isar:git:r0:${SSTATE_PKGARCH}:10:e794e52fe10ccf60b559089c39eaf86a4990
> a81991c9b1b0b2672721878a1b8b_prepare_build.tar.zst.siginfo'
> 
> 5. Remove stamps directory.
> 6. Build again - SSTATE should be used for the package but it is not 
> and the tasks are run again.
> 
> I think it is related to
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> ub.com%2Filbers%2Fisar%2Fblob%2F61086915e6c83fff22effa85cda64a2ac0c2f1
> 00%2Fmeta%2Fclasses%2Fsstate.bbclass%23L136&data=05%7C01%7Cjan.waltl%4
> 0siemens.com%7C8d77413c796a4920807b08dbc426e043%7C38ae3bcd95794fd4adda
> b42e1495d55a%7C1%7C0%7C638319443476294267%7CUnknown%7CTWFpbGZsb3d8eyJW
> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> 7C%7C%7C&sdata=GrO630mertcLooKbWBUDIS57z7VppVi1ntvVyayCOs4%3D&reserved
> =0
>  and BUILD_ARCH being empty according to 'bitbake -e'. 
> 
> I changed the line to HOST_ARCH (+expand=True) which seems to work but 
> I do not have enough expertise to guess its correctness. Maybe the 
> line in native.bbclass covers this "if statement" already?
> 
> How to best resolve this?
> 
> 
Hello, Jan.

I've tried to repeat your steps and reproduced the issue in case of cross-building.

Changing the line to HOST_ARCH is techincally correct. But since we don't like forking (sstate.bbclass is taken from some OE branch without any changes), I think it would be better to just define BUILD_ARCH somewhere else in Isar to the value of HOST_ARCH, for example:

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf index 9eb93e2b..0d737b64 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -90,6 +90,7 @@ NATIVELSBSTRING ?= "isarnative"
 TARGET_VENDOR = ""
 TARGET_OS = "isar"
 PACKAGE_ARCH ?= "${DISTRO_ARCH}"
+BUILD_ARCH ?= "${HOST_ARCH}"
 
 # Isar apt repository paths
 REPO_ISAR_DIR = "${DEPLOY_DIR}/isar-apt/${DISTRO}-${DISTRO_ARCH}/apt"

This seems to work and doesn't fork sstate.bbclass, so it can be later updated to some newer OE revision.

Unfortunately, HOST_ARCH has different meanings in Isar and Yocto...

> [This is my first message to this mailing list, please correct me if I 
> am doing something wrong]
> 
> 
> Kind Regards,
> Jan Waltl
> 
> Siemens Mobility, s.r.o.
> SMO RI LCE CZ SAF 1
> 
> 
>