Re: [PATCH 1/1] wic: disable generation of python bytecode cache

["'MOESSBAUER, Felix' via isar-users" <isar-users@googlegroups.com>]

On Fri, 2025-06-06 at 08:21 +0200, Jan Kiszka wrote:
> On 06.06.25 08:19, Heinisch, Alexander (FT RPD CED SES-AT) wrote:
> > On Thu, 2025-06-05 at 16:58 +0200, 'Felix Moessbauer' via isar-
> > users
> > wrote:
> > > Wic is executed as root, so the python bytecode cache is also
> > > created as root. This is problematic as the cache is created
> > > inside
> > > the
> > > source tree in a folder__pycache__ next to the python script
> > > itself.
> > > By
> > > that, we end up with files belonging to root inside the repo
> > > source
> > > tree, which makes it impossible to delete the source tree as a
> > > regular
> > > user.
> > > 
> > > This problem became visible with the kas purge plugin that
> > > removes
> > > the
> > > fetched layers as a regular user (these layers are fetched and
> > > managed
> > > by kas). Also the read-only mounting of repos in kas does not
> > > help
> > > here,
> > > as the fetched repos are not mounted ro for obvious reasons.
> > > 
> > > Anyways, we should not create files inside the source tree that
> > > do
> > > not
> > > belong to the calling user. To fix this, we just disable the
> > > python
> > > cache for the wic task. This is the only task that executes a
> > > python
> > > script from the source tree as root.
> > 
> > Hi Felix,
> > 
> > What about moving the cache to the build dir?
> > 
> > I just did a quick test using `PYTHONPYCACHEPREFIX` (see [1]) which
> > was
> > added in Python 3.8 (so works for hosts >= bullseye)
> > 
> > ```
> > diff --git a/meta/classes/imagetypes_wic.bbclass
> > b/meta/classes/imagetypes_wic.bbclass
> > index 38b5f0e1..7b8dc38c 100644
> > --- a/meta/classes/imagetypes_wic.bbclass
> > +++ b/meta/classes/imagetypes_wic.bbclass
> > @@ -157,6 +157,7 @@ generate_wic_image() {
> >      export FAKEROOTCMD=${FAKEROOTCMD}
> >      export BUILDDIR=${TOPDIR}
> >      export MTOOLS_SKIP_CHECK=1
> > +    export PYTHONPYCACHEPREFIX="${TOPDIR}/__pycache__"

This script is not called frequently, so there is no real benefit in
caching it. If we want to cache, we should probably cache in the
WORKDIR.

> >      mkdir -p ${IMAGE_ROOTFS}/../pseudo
> >      touch ${IMAGE_ROOTFS}/../pseudo/files.db```
> > 
> > and it seems to do the job. In depth testing still required!
> > 
> > [1]:
> > https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPYCACHEPREFIX
> > 
> 
> What's yocto doing about this BTW? Are we just missing some of their
> bits, or are they affected by a similar issue (except for creating
> the
> cache as root)?

Yocto also disables the python cache for their fakeroot environments:

https://github.com/openembedded/openembedded-core/blob/bd8fc4c59a137a37bd7a54f398949617982d447e/meta/conf/bitbake.conf#L753

But as isar differs here, we either need to add this statement where I
added it (take the patch as-is), or implement this logic in
`wic_fakeroot`.

Felix

> 
> Jan

-- 
Siemens AG
Linux Expert Center
Friedrich-Ludwig-Bauer-Str. 3
85748 Garching, Germany

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/a588fb53f96013a4adb2a35c503dbf09228fae4f.camel%40siemens.com.