[RFC PATCH 0/1] FILESEXTRAPATHS in .inc files

[RFC PATCH 0/1] FILESEXTRAPATHS in .inc files

[Claudius Heine]

Hi,

I was writing a kernel module recipe, and I notices that I cannot
directly overwrite files that where included by modules.inc.

Isar seems to use FILESEXTRAPATHS:prepend in .inc files in multiple
places, is there a reason for it?

Attached is a RFC patch to replace FILESEXTRAPATHS:prepend with
FILESPATH:append for demonstration of what I propose.

I can do this for the other .inc files as well, but I wanted to see if I
miss something here.

regards,
Claudius

Claudius Heine (1):
  linux-module: append local files path to FILSPATH

 meta/recipes-kernel/linux-module/module.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.40.1

[RFC PATCH 1/1] linux-module: append local files path to FILSPATH

[Claudius Heine]

FILESEXTRAPATHS is meant to allow bbappend files to overwrite files from
their recipe. For this reason the 'normal' patter in bbappend files is
to prepend the local file path to FILESEXTRAPATHS, which results in
files from the local file path to be preferred over files from the
original recipe, or other, lower prioritized bbappend files.

The `module.inc` file is meant to be included by other recipes (`.bb`)
files. So it should still allow the recipe to overwrite files its files
that where added via the normal recipes `FILESPATH` search path.

If `module.inc` uses `FILESEXTRAPATHS:prepend` to add its local file
path to `FILESPATH`, then the recipe, that includes the `module.inc`
would have to use `FILESEXTRAPATHS:prepend` as well, after including the
`module.inc` file in order to overwrite the files from the `module.inc`
file. However this is a workaround of the issue.

`module.inc` should append its local file search path to `FILESPATH`, so
that it allows other recipes to overwrite any file from it, without
requiring an additional workaround.

Signed-off-by: Claudius Heine <ch@denx.de>
---
 meta/recipes-kernel/linux-module/module.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index 76ad860..333c66b 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -5,7 +5,7 @@
 #
 # SPDX-License-Identifier: MIT
 
-FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/files:"
+FILESPATH:append := ":${FILE_DIRNAME}/files"
 
 DESCRIPTION ?= "Custom kernel module ${PN}"
 
-- 
2.40.1

Re: [RFC PATCH 1/1] linux-module: append local files path to FILSPATH

[Claudius Heine]

Hi Vijai,

I looked in the gitlog, and it seems like you last changed this stuff in 
`83bfec59f6e4b104de1acabecd51dd08d2c840d8`:

     Replace FILESPATH with FILESEXTRAPATHS

     The suggested approach to include files & patches from bbappend
     files is FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

     This is not possible if FILESPATH is modified directly. Use
     FILESEXTRAPATHS instead.

I an not 100% sure I understand you here. but couldn't this be solved by 
using the override syntax on `FILESPATH` instead of `=.`?

regards,
Claudius

On 2023-06-29 15:21, Claudius Heine wrote:
> FILESEXTRAPATHS is meant to allow bbappend files to overwrite files from
> their recipe. For this reason the 'normal' patter in bbappend files is
> to prepend the local file path to FILESEXTRAPATHS, which results in
> files from the local file path to be preferred over files from the
> original recipe, or other, lower prioritized bbappend files.
> 
> The `module.inc` file is meant to be included by other recipes (`.bb`)
> files. So it should still allow the recipe to overwrite files its files
> that where added via the normal recipes `FILESPATH` search path.
> 
> If `module.inc` uses `FILESEXTRAPATHS:prepend` to add its local file
> path to `FILESPATH`, then the recipe, that includes the `module.inc`
> would have to use `FILESEXTRAPATHS:prepend` as well, after including the
> `module.inc` file in order to overwrite the files from the `module.inc`
> file. However this is a workaround of the issue.
> 
> `module.inc` should append its local file search path to `FILESPATH`, so
> that it allows other recipes to overwrite any file from it, without
> requiring an additional workaround.
> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>   meta/recipes-kernel/linux-module/module.inc | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
> index 76ad860..333c66b 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -5,7 +5,7 @@
>   #
>   # SPDX-License-Identifier: MIT
>   
> -FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/files:"
> +FILESPATH:append := ":${FILE_DIRNAME}/files"
>   
>   DESCRIPTION ?= "Custom kernel module ${PN}"
>   

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

RE: [RFC PATCH 1/1] linux-module: append local files path to FILSPATH

[Kanagarajan, Vijaikumar]

-----Original Message-----
From: Claudius Heine <ch@denx.de> 
Sent: 29 June 2023 19:14
To: isar-users@googlegroups.com; Vijaikumar_Kanagarajan <Vijaikumar_Kanagarajan@mentor.com>
Subject: Re: [RFC PATCH 1/1] linux-module: append local files path to FILSPATH

Hi Vijai,

I looked in the gitlog, and it seems like you last changed this stuff in
`83bfec59f6e4b104de1acabecd51dd08d2c840d8`:

     Replace FILESPATH with FILESEXTRAPATHS

     The suggested approach to include files & patches from bbappend
     files is FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

     This is not possible if FILESPATH is modified directly. Use
     FILESEXTRAPATHS instead.

I an not 100% sure I understand you here. but couldn't this be solved by using the override syntax on `FILESPATH` instead of `=.`?

regards,
Claudius

On 2023-06-29 15:21, Claudius Heine wrote:
> FILESEXTRAPATHS is meant to allow bbappend files to overwrite files 
> from their recipe. For this reason the 'normal' patter in bbappend 
> files is to prepend the local file path to FILESEXTRAPATHS, which 
> results in files from the local file path to be preferred over files 
> from the original recipe, or other, lower prioritized bbappend files.
> 
> The `module.inc` file is meant to be included by other recipes (`.bb`) 
> files. So it should still allow the recipe to overwrite files its 
> files that where added via the normal recipes `FILESPATH` search path.
> 
> If `module.inc` uses `FILESEXTRAPATHS:prepend` to add its local file 
> path to `FILESPATH`, then the recipe, that includes the `module.inc` 
> would have to use `FILESEXTRAPATHS:prepend` as well, after including 
> the `module.inc` file in order to overwrite the files from the 
> `module.inc` file. However this is a workaround of the issue.
> 
> `module.inc` should append its local file search path to `FILESPATH`, 
> so that it allows other recipes to overwrite any file from it, without 
> requiring an additional workaround.

I don't see a better way than this.
We have an inc file which exposes a default dir. When we build a recipe the default FILESPATH is populated and now we have no mechanism to override those unless we do another FILESEXTRAPATHS:prepend after the include.
I assume this should be the case with other includes as well.

Acked-by: Vijai Kumar K <vijaikumar.kanagarajan@siemens.com>

Also wondering what kind of limitation was there in the current debian/ contents that you must override?

Thanks,
Vijai Kumar K

> 
> Signed-off-by: Claudius Heine <ch@denx.de>
> ---
>   meta/recipes-kernel/linux-module/module.inc | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/recipes-kernel/linux-module/module.inc 
> b/meta/recipes-kernel/linux-module/module.inc
> index 76ad860..333c66b 100644
> --- a/meta/recipes-kernel/linux-module/module.inc
> +++ b/meta/recipes-kernel/linux-module/module.inc
> @@ -5,7 +5,7 @@
>   #
>   # SPDX-License-Identifier: MIT
>   
> -FILESEXTRAPATHS:prepend := "${FILE_DIRNAME}/files:"
> +FILESPATH:append := ":${FILE_DIRNAME}/files"
>   
>   DESCRIPTION ?= "Custom kernel module ${PN}"
>   

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: [RFC PATCH 1/1] linux-module: append local files path to FILSPATH

[Claudius Heine]

Hi Vijai,

On 2023-06-30 13:36, 'Kanagarajan, Vijaikumar' via isar-users wrote:
> On 2023-06-29 15:21, Claudius Heine wrote:
>> FILESEXTRAPATHS is meant to allow bbappend files to overwrite files
>> from their recipe. For this reason the 'normal' patter in bbappend
>> files is to prepend the local file path to FILESEXTRAPATHS, which
>> results in files from the local file path to be preferred over files
>> from the original recipe, or other, lower prioritized bbappend files.
>>
>> The `module.inc` file is meant to be included by other recipes (`.bb`)
>> files. So it should still allow the recipe to overwrite files its
>> files that where added via the normal recipes `FILESPATH` search path.
>>
>> If `module.inc` uses `FILESEXTRAPATHS:prepend` to add its local file
>> path to `FILESPATH`, then the recipe, that includes the `module.inc`
>> would have to use `FILESEXTRAPATHS:prepend` as well, after including
>> the `module.inc` file in order to overwrite the files from the
>> `module.inc` file. However this is a workaround of the issue.
>>
>> `module.inc` should append its local file search path to `FILESPATH`,
>> so that it allows other recipes to overwrite any file from it, without
>> requiring an additional workaround.
> 
> I don't see a better way than this.
> We have an inc file which exposes a default dir. When we build a recipe the default FILESPATH is populated and now we have no mechanism to override those unless we do another FILESEXTRAPATHS:prepend after the include.
> I assume this should be the case with other includes as well.
> 
> Acked-by: Vijai Kumar K <vijaikumar.kanagarajan@siemens.com>
> 
> Also wondering what kind of limitation was there in the current debian/ contents that you must override?

Thanks for the ack. I will prepare a patchset that uses this scheme for 
the other FILESEXTRAPATHS in .inc files.

The issue with the debian files of modules.inc (and the split in isar 
between bitbake recipes and debian source packages) is that it contains 
the 'rules' file, which specifies how the Makefile of the project is 
executed. That means if the developer of the kernel module uses a 
different interface than `make -C $KDIR M=$(pwd) modules` to build the 
modules, then the `rules` file needs to be changed. This cannot be done 
by just appending to the recipe.

Sure, it would be possible to apply a patch to the source modules first, 
so that the same `rules` file can be used, but depending on how complex 
the build process of the out-of-tree kernel module is, this might be 
more complex, and less-maintainable than just replacing the `rules` file.

In my case it was about the out-of-tree nvidia tegra kernel modules. In 
some cases, they have creative ideas on how to do things.

regards,
Claudius

> 
> Thanks,
> Vijai Kumar K