[PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Felix Moessbauer]

The patch "fetch2/aws: forward env-vars used in gitlab-ci K8s" was
accepted upstream as c534526e. We need to backport it here to have
it already until the next bitbake LTS version is released:

This patch adds the following variables to the allow-list, which are
used in the "IAM roles for AWS when using the GitLab chart":

- AWS_ROLE_ARN
- AWS_WEB_IDENTITY_TOKEN_FILE

These variables are set in the CI job environment and are needed to
access the sstate cache artifacts in a connected S3 bucket.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 bitbake/lib/bb/fetch2/__init__.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/bitbake/lib/bb/fetch2/__init__.py b/bitbake/lib/bb/fetch2/__init__.py
index a3140626..008adab9 100644
--- a/bitbake/lib/bb/fetch2/__init__.py
+++ b/bitbake/lib/bb/fetch2/__init__.py
@@ -850,6 +850,8 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH',
                      'AWS_PROFILE',
                      'AWS_ACCESS_KEY_ID',
                      'AWS_SECRET_ACCESS_KEY',
+                     'AWS_ROLE_ARN',
+                     'AWS_WEB_IDENTITY_TOKEN_FILE',
                      'AWS_DEFAULT_REGION']
 
 def get_fetcher_environment(d):
-- 
2.39.2

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Uladzimir Bely]

On Fri, 2023-11-24 at 15:50 +0800, 'Felix Moessbauer' via isar-users
wrote:
> The patch "fetch2/aws: forward env-vars used in gitlab-ci K8s" was
> accepted upstream as c534526e. We need to backport it here to have
> it already until the next bitbake LTS version is released:
> 
> This patch adds the following variables to the allow-list, which are
> used in the "IAM roles for AWS when using the GitLab chart":
> 
> - AWS_ROLE_ARN
> - AWS_WEB_IDENTITY_TOKEN_FILE
> 
> These variables are set in the CI job environment and are needed to
> access the sstate cache artifacts in a connected S3 bucket.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/bitbake/lib/bb/fetch2/__init__.py
> b/bitbake/lib/bb/fetch2/__init__.py
> index a3140626..008adab9 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -850,6 +850,8 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH',
>                       'AWS_PROFILE',
>                       'AWS_ACCESS_KEY_ID',
>                       'AWS_SECRET_ACCESS_KEY',
> +                     'AWS_ROLE_ARN',
> +                     'AWS_WEB_IDENTITY_TOKEN_FILE',
>                       'AWS_DEFAULT_REGION']
>  
>  def get_fetcher_environment(d):
> -- 
> 2.39.2
> 

It looks like a fork of bitbake-2.0.12 we currently use in Isar that we
usually tend to avoid. But, probably not at this time...

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Baurzhan Ismagulov]

On 2023-11-24 16:52, Uladzimir Bely wrote:
> > --- a/bitbake/lib/bb/fetch2/__init__.py
> > +++ b/bitbake/lib/bb/fetch2/__init__.py
> > @@ -850,6 +850,8 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH',
> > ��������������������� 'AWS_PROFILE',
> > ��������������������� 'AWS_ACCESS_KEY_ID',
> > ��������������������� 'AWS_SECRET_ACCESS_KEY',
> > +�������������������� 'AWS_ROLE_ARN',
> > +�������������������� 'AWS_WEB_IDENTITY_TOKEN_FILE',
> > ��������������������� 'AWS_DEFAULT_REGION']
> > �
> > �def get_fetcher_environment(d):
> 
> It looks like a fork of bitbake-2.0.12 we currently use in Isar that we
> usually tend to avoid. But, probably not at this time...

If that is already upstream, I think it should be pretty straightforward to
backport this till we get 2.6.1 or so.

With kind regards,
Baurzhan

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Jan Kiszka]

On 24.11.23 22:00, Baurzhan Ismagulov wrote:
> On 2023-11-24 16:52, Uladzimir Bely wrote:
>>> --- a/bitbake/lib/bb/fetch2/__init__.py
>>> +++ b/bitbake/lib/bb/fetch2/__init__.py
>>> @@ -850,6 +850,8 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH',
>>>                       'AWS_PROFILE',
>>>                       'AWS_ACCESS_KEY_ID',
>>>                       'AWS_SECRET_ACCESS_KEY',
>>> +                     'AWS_ROLE_ARN',
>>> +                     'AWS_WEB_IDENTITY_TOKEN_FILE',
>>>                       'AWS_DEFAULT_REGION']
>>>  
>>>  def get_fetcher_environment(d):
>>
>> It looks like a fork of bitbake-2.0.12 we currently use in Isar that we
>> usually tend to avoid. But, probably not at this time...
> 
> If that is already upstream, I think it should be pretty straightforward to
> backport this till we get 2.6.1 or so.
> 

As discussed with Felix already in person: The only issue would be if we
decided to update once more in the current stable series. Then there is
the risk that we forget about this patch and overwrite it because
upstream stable will not see it (it's a feature, not a fix). Or is 2.0.x
EOL anyway?

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Uladzimir Bely]

On Fri, 2023-11-24 at 15:50 +0800, 'Felix Moessbauer' via isar-users
wrote:
> The patch "fetch2/aws: forward env-vars used in gitlab-ci K8s" was
> accepted upstream as c534526e. We need to backport it here to have
> it already until the next bitbake LTS version is released:
> 
> This patch adds the following variables to the allow-list, which are
> used in the "IAM roles for AWS when using the GitLab chart":
> 
> - AWS_ROLE_ARN
> - AWS_WEB_IDENTITY_TOKEN_FILE
> 
> These variables are set in the CI job environment and are needed to
> access the sstate cache artifacts in a connected S3 bucket.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  bitbake/lib/bb/fetch2/__init__.py | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/bitbake/lib/bb/fetch2/__init__.py
> b/bitbake/lib/bb/fetch2/__init__.py
> index a3140626..008adab9 100644
> --- a/bitbake/lib/bb/fetch2/__init__.py
> +++ b/bitbake/lib/bb/fetch2/__init__.py
> @@ -850,6 +850,8 @@ FETCH_EXPORT_VARS = ['HOME', 'PATH',
>                       'AWS_PROFILE',
>                       'AWS_ACCESS_KEY_ID',
>                       'AWS_SECRET_ACCESS_KEY',
> +                     'AWS_ROLE_ARN',
> +                     'AWS_WEB_IDENTITY_TOKEN_FILE',
>                       'AWS_DEFAULT_REGION']
>  
>  def get_fetcher_environment(d):
> -- 
> 2.39.2
> 

Applied to next, thanks.

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Baurzhan Ismagulov]

On 2023-11-25 16:19, 'Jan Kiszka' via isar-users wrote:
> As discussed with Felix already in person: The only issue would be if we
> decided to update once more in the current stable series. Then there is
> the risk that we forget about this patch and overwrite it because
> upstream stable will not see it (it's a feature, not a fix). Or is 2.0.x
> EOL anyway?

I think that we as maintainers should take care of forward-porting the change
whenever we update bitbake. Currently, we use 2.0 due to the LTS. We estimate
that the upstream change might become part of 2.6.1, which would mean we might
still have 2.4, 2.6 and / or 2.8 (which is the next LTS).

With kind regards,
Baurzhan

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[MOESSBAUER, Felix]

On Wed, 2023-11-29 at 11:11 +0100, Baurzhan Ismagulov wrote:
> On 2023-11-25 16:19, 'Jan Kiszka' via isar-users wrote:
> > As discussed with Felix already in person: The only issue would be
> > if we
> > decided to update once more in the current stable series. Then
> > there is
> > the risk that we forget about this patch and overwrite it because
> > upstream stable will not see it (it's a feature, not a fix). Or is
> > 2.0.x
> > EOL anyway?
> 
> I think that we as maintainers should take care of forward-porting
> the change
> whenever we update bitbake. Currently, we use 2.0 due to the LTS. We
> estimate
> that the upstream change might become part of 2.6.1, which would mean
> we might
> still have 2.4, 2.6 and / or 2.8 (which is the next LTS).

Hi, this sounds reasonable. Is there any documentation, where the
backported patches are tracked?

Best regards,
Felix

> 
> With kind regards,
> Baurzhan
> 

Re: [PATCH 1/1] bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s

[Baurzhan Ismagulov]

On 2023-11-30 02:48, MOESSBAUER, Felix wrote:
> > I think that we as maintainers should take care of forward-porting
> > the change
> > whenever we update bitbake. Currently, we use 2.0 due to the LTS. We
> > estimate
> > that the upstream change might become part of 2.6.1, which would mean
> > we might
> > still have 2.4, 2.6 and / or 2.8 (which is the next LTS).
> 
> Hi, this sounds reasonable. Is there any documentation, where the
> backported patches are tracked?

We've thought about introducing bitbake/README.Isar, but it would still be
something that one has to actively check, which is then not much better than
e.g. git log --pretty=oneline --abbrev-commit --no-decorate bitbake:

2253ad4d bitbake: backport fetch2/aws: forward env-vars used in gitlab-ci K8s
05710647 bitbake: update to Bitbake 2.0.12
a9bbdeeb bitbake: update to Bitbake 2.0.5
2a10bbb9 utils: Ensure shell function failure in python logging is correct
01084712 bitbake-diffsigs: break on first dependent task difference
a5bbc86f bitbake-diffsigs: make finding of changed signatures more robust
3c99609b bitbake: Update to 1.50.5 release
c9252baf bitbake: Update to 1.50.4 release
522ac8f8 bitbake: Update to 1.46.2 release
c2aae245 bitbake: update to version 1.44.0
ce34a501 bitbake: Update to fixed master revision
1fb4c991 bitbake: Update to the release 1.40.0
a6e101f5 Update bitbake from the upstream.
3a0254fc bitbake: Update to 1.31.2+g80f377e
ff22b2b1 Add bitbake master 7c849be

The better alternative would be a testcase.

With kind regards,
Baurzhan