Re: [RFC] Documentation: dependencies specification in Bitbake and Debian

["Cirujano Cuesta, Silvano" <silvano.cirujano-cuesta@siemens.com>]

Hi Claudius,

First of all thank you very much for your quick and valuable feedback!

On Wed, 2019-02-06 at 16:00 +0100, Claudius Heine wrote:
> > ## Dependencies table
> > 
> > Acronyms:
> > > Bt = Buildtime
> > > 
> > > Rt = Runtime
> > > Variable           | Bt Bitbake | Bt Debian | Rt Functional | Rt Package |
> > > ------------------ |:----------:|:---------:|:-------------:|:----------:|
> > > `DEPENDS`          |     x      |           |               |            |
> > > `IMAGE_INSTALL`    |     x      |           |      x        |            | > | `IMAGE_PREINSTALL` 
> > 
> >             |           |      x        |            |
> > > `DEBIAN_DEPENDS`   |            |           |               |     x      |
> 
> While this graphic looks very simple, that is misguiding.
> IMO they could lead to a viewpoint that makes really understanding what 
> is happening more difficult. Bitbake works its dependency magic on task 
> level and provides DEPENDS to express common task dependencies across 
> recipes as syntax sugar.
> 
> IMAGE_INSTALL + IMAGE_PREINSTALL + DEBIAN_DEPENDS etc. is more about 
> expressing debian package installation and a bit more. This time it is 
> isar syntax sugar that puts the entries from IMAGE_INSTALL into DEPENDS.
> 
> IMO this sugar, while it makes some things more easy (with big 
> assumptions), makes some things more difficult (recipes provide multiple 
> packages). AFAIK this wasn't changed because historic reasons.

I'm not sure I get your point here...

My interpretation of your comment is that the table is right, but you would prefer the table not to show the syntactic sugar.
Mostly to avoid people getting into anti-patterns, I assume.
Did I get it right?

The separation between Bitbake dependency at task level and Debian dependency at package level is clear to me, perhaps it's only the presentation of the table that is misleading...

> > ### Bitbake buildtime: `DEPENDS`
> > 
> > `DEPENDS` specifies recipes available in one of the layers that have to be built.
> > 
> > The logical place to use this variable depends on which kind of dependency it should specify.
> > 
> > Is it a recipe that builds a package that requires another custom package (built from a Bitbake recipe)?
> > Then probably a **package** recipe is the best place. > The [above table](#dependencies-table) shows that there is no way to 
> 
> specify at the same time a Bitbake runtime dependency and a Debian 
> Package dependency, therefore the corresponding [package runtime
> > dependency](#package-runtime-dependencies) should be also added.
> > 
> > Otherwise probably an **image** recipe or **configuration** file is the best place.
> 
> No. DEPENDS in a configuration file does not make any sense to me. 
> Configuration files are global, that would mean every recipes would have 
> this dependency now (if it doesn't just overwrite it)

You are right. I will fix it.

> 
> And for Isar DEPENDS in a image recipe is very obscure as well, see 
> IMAGER_BUILD_DEPS, where it is used.

WHAT?! I haven't found any reference to `IMAGER_BUILD_DEPS` anywhere in the documentation!

Looking at the code [1] I see that it's an obvious miss in the documentation :-)
Damn it, I'll have to analyze that code if I want this documentation to be complete...

[1] https://github.com/ilbers/isar/blob/6c5db020b9b837d7b0ce63bfc719f9192e725f26/meta/classes/image.bbclass#L131

You are anyway right, that an image recipe is an obscure place for a DEPENDS.
The right place would be a package recipe. I will fix it.

> 
> DEPENDS also has nothing to do with packages directly but with the vague 
> dependencies between recipes that then get concrete transferred into 
> dependencies of tasks between recipes.
> 
> See:
> 
> https://www.yoctoproject.org/docs/latest/bitbake-user-manual/bitbake-user-manual.html#build-dependencies

That's clear to me, apparently the formulation in the text doesn't express it clearly enough.

> 
> > Opposed to `IMAGE_INSTALL`, this doesn't affect any runtime dependencies.
> > 
> > This variable can be seen as the counterpart at buildtime for [`IMAGE_PREINSTALL`](#functional-runtime-image_preinstall) (runtime) and [`DEBIAN_DEPENDS`](#package-runtime-debian_depends)
> > (runtime).
> > 
> > ### Bitbake buildtime: `IMAGE_INSTALL`
> > 
> > `IMAGE_INSTALL` specifies recipes available in one of the layers that have to be built. > And at the same time specifies that the Debian packages resulting 
> 
> from the recipes have to be installed in the image (see 
> [below](#functional-runtime-image_install) for more information).
> > 
> > The logical place to use this variable is an **image** recipe or **configuration** file (distro, layer, ...).
> 
> The only places where this is possible.

Does "is possible" really mean that either you get an error or it's being silently ignored?
Or there is no enforcement, but it shouldn't be done elsewhere?

> 
> Speaking of images, there are also 'debian build dependencies' for 
> images called 'IMAGER_INSTALL'.

Another surprise! I haven't found any reference either to it anywhere in the documentation!
The same as `IMAGER_BUILD_DEPS`.

> 
> > 
> > ## How to control runtime dependencies
> > 
> > Runtime dependencies specify which Debian packages are made availabe during runtime (by installing them in the image, if one is being created).
> > 
> > The union of packages specified via `IMAGE_INSTALL`, `IMAGE_PREINSTALL`, `DEBIAN_DEPENDS`, `debian/control` files of custom packages and dependencies of upstream Debian packages get installed into
> > the
> > image.
> 
> + the minbase that debootstrap installs.

True. I will add it.

> > ### Package runtime: `DEBIAN_DEPENDS`
> > 
> > `DEBIAN_DEPENDS` specifies the packages that are required by the package being built.
> > They will be added to the [`Depends` of the package control file][pkg-rels] and therefore have to be specified in the format expected by Debian.
> > 
> > The logical place to use this variable is a **package** recipe.
> 
> Only place are dpkg-raw packages that are package recipes.

Right. Too much copy&paste :-)

> > ### `RRECOMMENDS`
> > 
> > This variable provides ["a list of packages that extends the usability of a package"][bb-rrecommends].
> > 
> > In ISAR there is only one way to achieve the same:
> > 
> > 1. Adding them to the [`Recommends` of the package `debian/control`][pkg-rels] file.
> 
> You can do this, but recommended or suggested packages are not installed 
> per default, but you could change that by overwriting the isar-apt.conf 
> of the isar-bootstrap-*.bb recipes.

That's clear to me in the case of Debian.
I've also assumed that the same applies to Yocto/OE

> 
> > 
> > ### `PACKAGES`
> > 
> > This variable provides ["the list of packages the recipe creates"][bb-packages].
> > 
> > In ISAR there is only one way to achieve the same:
> > 
> > 1. Specifying multiple packages in the [`debian/control` file][pkg-control] being provided by a recipe.
> 
> Doing so might get problematic because you then cannot use IMAGE_INSTALL 
> to install those specific packages, because they then will be added to 
> DEPENDS (as I said IMO bad idea) and have to remove those from DEPENDS 
> (DEPENDS_remove) and add the correct recipe name manually.

Well, `DEPENDS` would be needed to get the "Buildtime dependency" (to ensure that the recipe gets executed and the Debian packages get built) and the control file would take care of the "Runtime
dependency" (to get the functionality in the image or a dependency of a package resolved).

I'm not sure what you mean it's a bad idea.
Hopefully you don't mean that having `DEPENDS` in a package recipe is a bad idea, since that's what the example does [2].

[2] https://github.com/ilbers/isar/blob/6c5db020b9b837d7b0ce63bfc719f9192e725f26/meta-isar/recipes-app/example-hello/example-hello.bb#L15

Regards,
  Silvano