[PATCH] bitbake: Allow setting DL_DIR from environment

[PATCH] bitbake: Allow setting DL_DIR from environment

[Harald Seiler]

This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
and then specifying a custom DL_DIR in the environment.

This behavior mirrors the one in poky.

Signed-off-by: Harald Seiler <hws@denx.de>
---
 bitbake/conf/bitbake.conf   | 2 +-
 meta/conf/isar-bitbake.conf | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf
index a460df4..ba0d68b 100644
--- a/bitbake/conf/bitbake.conf
+++ b/bitbake/conf/bitbake.conf
@@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs"
 DEPENDS = ""
 DEPLOY_DIR = "${TMPDIR}/deploy"
 DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images"
-DL_DIR = "${TMPDIR}/downloads"
+DL_DIR ?= "${TMPDIR}/downloads"
 FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
 FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
 GITDIR = "${DL_DIR}/git"
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index 666c4a3..39e73f6 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -19,7 +19,7 @@
 # OTHER DEALINGS IN THE SOFTWARE.
 
 WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
-DL_DIR = "${TOPDIR}/downloads"
+DL_DIR ?= "${TOPDIR}/downloads"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
 BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
 BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
--
2.14.1

Re: [PATCH] bitbake: Allow setting DL_DIR from environment

[chombourger]

[-- Attachment #1.1: Type: text/plain, Size: 1699 bytes --]

Looks good to me

On Tuesday, October 30, 2018 at 5:44:14 PM UTC+1, Harald Seiler wrote:
>
> This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE 
> and then specifying a custom DL_DIR in the environment. 
>
> This behavior mirrors the one in poky. 
>
> Signed-off-by: Harald Seiler <h...@denx.de <javascript:>> 
> --- 
>  bitbake/conf/bitbake.conf   | 2 +- 
>  meta/conf/isar-bitbake.conf | 2 +- 
>  2 files changed, 2 insertions(+), 2 deletions(-) 
>
> diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf 
> index a460df4..ba0d68b 100644 
> --- a/bitbake/conf/bitbake.conf 
> +++ b/bitbake/conf/bitbake.conf 
> @@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs" 
>  DEPENDS = "" 
>  DEPLOY_DIR = "${TMPDIR}/deploy" 
>  DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images" 
> -DL_DIR = "${TMPDIR}/downloads" 
> +DL_DIR ?= "${TMPDIR}/downloads" 
>  FILESPATH = 
> "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}" 
>
>  FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}" 
>  GITDIR = "${DL_DIR}/git" 
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf 
> index 666c4a3..39e73f6 100644 
> --- a/meta/conf/isar-bitbake.conf 
> +++ b/meta/conf/isar-bitbake.conf 
> @@ -19,7 +19,7 @@ 
>  # OTHER DEALINGS IN THE SOFTWARE. 
>   
>  WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}" 
> -DL_DIR = "${TOPDIR}/downloads" 
> +DL_DIR ?= "${TOPDIR}/downloads" 
>  SSTATE_DIR ?= "${TMPDIR}/sstate-cache" 
>  BUILDCHROOT_HOST_DIR = 
> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs" 
>  BUILDCHROOT_TARGET_DIR = 
> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs" 
> -- 
> 2.14.1 
>
>

[-- Attachment #1.2: Type: text/html, Size: 2347 bytes --]

Re: [PATCH] bitbake: Allow setting DL_DIR from environment

[Jan Kiszka]

On 30.10.18 17:44, Harald Seiler wrote:
> This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
> and then specifying a custom DL_DIR in the environment.
> 
> This behavior mirrors the one in poky.
> 
> Signed-off-by: Harald Seiler <hws@denx.de>
> ---
>   bitbake/conf/bitbake.conf   | 2 +-

We do not patch bitbake so far. Is this fix in upstream already? Then let's 
fast-forward bitbake to that revision. If not, send it upstream first. Local, 
isar-specific changes go to isar-bitbake.conf.

Jan

>   meta/conf/isar-bitbake.conf | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf
> index a460df4..ba0d68b 100644
> --- a/bitbake/conf/bitbake.conf
> +++ b/bitbake/conf/bitbake.conf
> @@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs"
>   DEPENDS = ""
>   DEPLOY_DIR = "${TMPDIR}/deploy"
>   DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images"
> -DL_DIR = "${TMPDIR}/downloads"
> +DL_DIR ?= "${TMPDIR}/downloads"
>   FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
>   FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
>   GITDIR = "${DL_DIR}/git"
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> index 666c4a3..39e73f6 100644
> --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -19,7 +19,7 @@
>   # OTHER DEALINGS IN THE SOFTWARE.
>   
>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
> -DL_DIR = "${TOPDIR}/downloads"
> +DL_DIR ?= "${TOPDIR}/downloads"
>   SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
>   BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
>   BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
> --
> 2.14.1
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Upgrading bitbake in isar

[Maxim Yu. Osipov]

Hi everybody,

I had several times requests regarding bitbake's upgrade in isar.
Do you have any suggestions about bitbake's revision to fast-forward?

Kind regards,
Maxim.

-------- Forwarded Message --------
Subject: Re: [PATCH] bitbake: Allow setting DL_DIR from environment
Date: Wed, 31 Oct 2018 07:39:01 +0100
From: Jan Kiszka <jan.kiszka@siemens.com>
To: Harald Seiler <hws@denx.de>, isar-users@googlegroups.com

On 30.10.18 17:44, Harald Seiler wrote:
> This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
> and then specifying a custom DL_DIR in the environment.
> 
> This behavior mirrors the one in poky.
> 
> Signed-off-by: Harald Seiler <hws@denx.de>
> ---
>   bitbake/conf/bitbake.conf   | 2 +-

We do not patch bitbake so far. Is this fix in upstream already? Then 
let's fast-forward bitbake to that revision. If not, send it upstream 
first. Local, isar-specific changes go to isar-bitbake.conf.

Jan

>   meta/conf/isar-bitbake.conf | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf
> index a460df4..ba0d68b 100644
> --- a/bitbake/conf/bitbake.conf
> +++ b/bitbake/conf/bitbake.conf
> @@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs"
>   DEPENDS = ""
>   DEPLOY_DIR = "${TMPDIR}/deploy"
>   DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images"
> -DL_DIR = "${TMPDIR}/downloads"
> +DL_DIR ?= "${TMPDIR}/downloads"
>   FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
>   FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
>   GITDIR = "${DL_DIR}/git"
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> index 666c4a3..39e73f6 100644
> --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -19,7 +19,7 @@
>   # OTHER DEALINGS IN THE SOFTWARE.
>   
>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
> -DL_DIR = "${TOPDIR}/downloads"
> +DL_DIR ?= "${TOPDIR}/downloads"
>   SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
>   BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
>   BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
> --
> 2.14.1
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

-- 
You received this message because you are subscribed to the Google 
Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to isar-users+unsubscribe@googlegroups.com.
To post to this group, send email to isar-users@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/isar-users/bad4755f-4246-9d22-f981-0847e7dacb93%40siemens.com.
For more options, visit https://groups.google.com/d/optout.

Re: Upgrading bitbake in isar

[Henning Schild]

Hi,

i already did that a few months ago and just went for latest-greatest.
And as expected that worked without much headache, after all it is only
our build tool which seems to be mostly backward compatible.
I remember a few warnings that would probably have required recipe
updates to silence them, but it did build on the first try (even the
whole thing in your CI). After that i spent some time on wic, which was
not that easy.

So i suggest "1.40". If you hit any blocking points a lower version
might also be ok, but i would not expect any. The trickier thing would
be to make Isar properly use the new bitbake after the bump. (using new
features, removing workarounds and alike).

Henning

Am Thu, 1 Nov 2018 13:45:20 +0300
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Hi everybody,
> 
> I had several times requests regarding bitbake's upgrade in isar.
> Do you have any suggestions about bitbake's revision to fast-forward?
> 
> Kind regards,
> Maxim.
> 
> -------- Forwarded Message --------
> Subject: Re: [PATCH] bitbake: Allow setting DL_DIR from environment
> Date: Wed, 31 Oct 2018 07:39:01 +0100
> From: Jan Kiszka <jan.kiszka@siemens.com>
> To: Harald Seiler <hws@denx.de>, isar-users@googlegroups.com
> 
> On 30.10.18 17:44, Harald Seiler wrote:
> > This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
> > and then specifying a custom DL_DIR in the environment.
> > 
> > This behavior mirrors the one in poky.
> > 
> > Signed-off-by: Harald Seiler <hws@denx.de>
> > ---
> >   bitbake/conf/bitbake.conf   | 2 +-  
> 
> We do not patch bitbake so far. Is this fix in upstream already? Then 
> let's fast-forward bitbake to that revision. If not, send it upstream 
> first. Local, isar-specific changes go to isar-bitbake.conf.
> 
> Jan
> 
> >   meta/conf/isar-bitbake.conf | 2 +-
> >   2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf
> > index a460df4..ba0d68b 100644
> > --- a/bitbake/conf/bitbake.conf
> > +++ b/bitbake/conf/bitbake.conf
> > @@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs"
> >   DEPENDS = ""
> >   DEPLOY_DIR = "${TMPDIR}/deploy"
> >   DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images"
> > -DL_DIR = "${TMPDIR}/downloads"
> > +DL_DIR ?= "${TMPDIR}/downloads"
> >   FILESPATH =
> > "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
> > FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
> > GITDIR = "${DL_DIR}/git" diff --git a/meta/conf/isar-bitbake.conf
> > b/meta/conf/isar-bitbake.conf index 666c4a3..39e73f6 100644
> > --- a/meta/conf/isar-bitbake.conf
> > +++ b/meta/conf/isar-bitbake.conf
> > @@ -19,7 +19,7 @@
> >   # OTHER DEALINGS IN THE SOFTWARE.
> >   
> >   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
> > -DL_DIR = "${TOPDIR}/downloads"
> > +DL_DIR ?= "${TOPDIR}/downloads"
> >   SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
> >   BUILDCHROOT_HOST_DIR =
> > "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
> > BUILDCHROOT_TARGET_DIR =
> > "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
> > -- 2.14.1 
> 

Re: [PATCH] bitbake: Allow setting DL_DIR from environment

[Harald Seiler]

On Wed, 2018-10-31 at 07:39 +0100, Jan Kiszka wrote:
> On 30.10.18 17:44, Harald Seiler wrote:
> > This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
> > and then specifying a custom DL_DIR in the environment.
> > 
> > This behavior mirrors the one in poky.
> > 
> > Signed-off-by: Harald Seiler <hws@denx.de>
> > ---
> >   bitbake/conf/bitbake.conf   | 2 +-
> 
> We do not patch bitbake so far. Is this fix in upstream already? Then let's 
> fast-forward bitbake to that revision. If not, send it upstream first. Local, 
> isar-specific changes go to isar-bitbake.conf.

Upstream (bitbake) does not have this fix.  The reason it works in poky is that
they have removed the `bitbake/conf/bitbake.conf` file completely and instead
use their own config in `meta/conf/bitbake.conf`.

I am not sure why this is done this way, though.

> 
> >   meta/conf/isar-bitbake.conf | 2 +-
> >   2 files changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/bitbake/conf/bitbake.conf b/bitbake/conf/bitbake.conf
> > index a460df4..ba0d68b 100644
> > --- a/bitbake/conf/bitbake.conf
> > +++ b/bitbake/conf/bitbake.conf
> > @@ -23,7 +23,7 @@ CVSDIR = "${DL_DIR}/cvs"
> >   DEPENDS = ""
> >   DEPLOY_DIR = "${TMPDIR}/deploy"
> >   DEPLOY_DIR_IMAGE = "${DEPLOY_DIR}/images"
> > -DL_DIR = "${TMPDIR}/downloads"
> > +DL_DIR ?= "${TMPDIR}/downloads"
> >   FILESPATH = "${FILE_DIRNAME}/${PF}:${FILE_DIRNAME}/${P}:${FILE_DIRNAME}/${PN}:${FILE_DIRNAME}/files:${FILE_DIRNAME}"
> >   FILE_DIRNAME = "${@os.path.dirname(d.getVar('FILE', False))}"
> >   GITDIR = "${DL_DIR}/git"
> > diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> > index 666c4a3..39e73f6 100644
> > --- a/meta/conf/isar-bitbake.conf
> > +++ b/meta/conf/isar-bitbake.conf
> > @@ -19,7 +19,7 @@
> >   # OTHER DEALINGS IN THE SOFTWARE.
> >   
> >   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
> > -DL_DIR = "${TOPDIR}/downloads"
> > +DL_DIR ?= "${TOPDIR}/downloads"
> >   SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
> >   BUILDCHROOT_HOST_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-host/rootfs"
> >   BUILDCHROOT_TARGET_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot-target/rootfs"
> > --
> > 2.14.1
> > 
> 

-- 
Harald

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-62  Fax: +49-8142-66989-80   Email: hws@denx.de 

Re: [PATCH] bitbake: Allow setting DL_DIR from environment

[Jan Kiszka]

On 06.11.18 12:06, Harald Seiler wrote:
> On Wed, 2018-10-31 at 07:39 +0100, Jan Kiszka wrote:
>> On 30.10.18 17:44, Harald Seiler wrote:
>>> This patch allows adding DL_DIR to BB_ENV_EXTRAWHITE
>>> and then specifying a custom DL_DIR in the environment.
>>>
>>> This behavior mirrors the one in poky.
>>>
>>> Signed-off-by: Harald Seiler <hws@denx.de>
>>> ---
>>>    bitbake/conf/bitbake.conf   | 2 +-
>>
>> We do not patch bitbake so far. Is this fix in upstream already? Then let's
>> fast-forward bitbake to that revision. If not, send it upstream first. Local,
>> isar-specific changes go to isar-bitbake.conf.
> 
> Upstream (bitbake) does not have this fix.  The reason it works in poky is that
> they have removed the `bitbake/conf/bitbake.conf` file completely and instead
> use their own config in `meta/conf/bitbake.conf`.
> 
> I am not sure why this is done this way, though.
> 

Then I would say - in order to keep bitbake/ clean - we should promote 
isar-bitbake.conf to the only bitbake.conf we use, migrating current 
bitbake.conf content over.

But maybe it is still worth try to convince upstream bitbake of this tiny change.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] bitbake: Allow setting DL_DIR from environment

[Baurzhan Ismagulov]

On Tue, Nov 06, 2018 at 12:06:10PM +0100, Harald Seiler wrote:
> Upstream (bitbake) does not have this fix.  The reason it works in poky is that
> they have removed the `bitbake/conf/bitbake.conf` file completely and instead
> use their own config in `meta/conf/bitbake.conf`.

This is how it was originally implemented in Isar, due to the fact that
upstream uses it that way. Later, this was changed with the goal of keeping
Isar modifications thin and ease bitbake upgrades. Of course, this will only
work if the upstream plays along.


> I am not sure why this is done this way, though.

I have an impression that the distinction "bitbake vs. OE / Yocto" upstream is
blurred and doesn't play big role, since the latter is being assumed the single
user. It would indeed be interesting to submit the change upstream and see the
feedback.


With kind regards,
Baurzhan.