Re: [PATCH 1/1] wic: disable generation of python bytecode cache

["'Jan Kiszka' via isar-users" <isar-users@googlegroups.com>]

On 06.06.25 09:18, Moessbauer, Felix (FT RPD CED OES-DE) wrote:
> On Fri, 2025-06-06 at 08:21 +0200, Jan Kiszka wrote:
>> On 06.06.25 08:19, Heinisch, Alexander (FT RPD CED SES-AT) wrote:
>>> On Thu, 2025-06-05 at 16:58 +0200, 'Felix Moessbauer' via isar-
>>> users
>>> wrote:
>>>> Wic is executed as root, so the python bytecode cache is also
>>>> created as root. This is problematic as the cache is created
>>>> inside
>>>> the
>>>> source tree in a folder__pycache__ next to the python script
>>>> itself.
>>>> By
>>>> that, we end up with files belonging to root inside the repo
>>>> source
>>>> tree, which makes it impossible to delete the source tree as a
>>>> regular
>>>> user.
>>>>
>>>> This problem became visible with the kas purge plugin that
>>>> removes
>>>> the
>>>> fetched layers as a regular user (these layers are fetched and
>>>> managed
>>>> by kas). Also the read-only mounting of repos in kas does not
>>>> help
>>>> here,
>>>> as the fetched repos are not mounted ro for obvious reasons.
>>>>
>>>> Anyways, we should not create files inside the source tree that
>>>> do
>>>> not
>>>> belong to the calling user. To fix this, we just disable the
>>>> python
>>>> cache for the wic task. This is the only task that executes a
>>>> python
>>>> script from the source tree as root.
>>>
>>> Hi Felix,
>>>
>>> What about moving the cache to the build dir?
>>>
>>> I just did a quick test using `PYTHONPYCACHEPREFIX` (see [1]) which
>>> was
>>> added in Python 3.8 (so works for hosts >= bullseye)
>>>
>>> ```
>>> diff --git a/meta/classes/imagetypes_wic.bbclass
>>> b/meta/classes/imagetypes_wic.bbclass
>>> index 38b5f0e1..7b8dc38c 100644
>>> --- a/meta/classes/imagetypes_wic.bbclass
>>> +++ b/meta/classes/imagetypes_wic.bbclass
>>> @@ -157,6 +157,7 @@ generate_wic_image() {
>>>      export FAKEROOTCMD=${FAKEROOTCMD}
>>>      export BUILDDIR=${TOPDIR}
>>>      export MTOOLS_SKIP_CHECK=1
>>> +    export PYTHONPYCACHEPREFIX="${TOPDIR}/__pycache__"
> 
> This script is not called frequently, so there is no real benefit in
> caching it. If we want to cache, we should probably cache in the
> WORKDIR.
> 
>>>      mkdir -p ${IMAGE_ROOTFS}/../pseudo
>>>      touch ${IMAGE_ROOTFS}/../pseudo/files.db```
>>>
>>> and it seems to do the job. In depth testing still required!
>>>
>>> [1]:
>>> https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPYCACHEPREFIX
>>>
>>
>> What's yocto doing about this BTW? Are we just missing some of their
>> bits, or are they affected by a similar issue (except for creating
>> the
>> cache as root)?
> 
> Yocto also disables the python cache for their fakeroot environments:
> 
> https://github.com/openembedded/openembedded-core/blob/bd8fc4c59a137a37bd7a54f398949617982d447e/meta/conf/bitbake.conf#L753
> 
> But as isar differs here, we either need to add this statement where I
> added it (take the patch as-is), or implement this logic in
> `wic_fakeroot`.

We have more python code than wic in isar. You should then make sure to
disable caching in general, not just for wic.

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/ff6c120d-a108-4001-bc60-c64af2d4f2d3%40siemens.com.