From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7343661073770217472 Date: Mon, 18 Mar 2024 06:58:23 -0700 (PDT) From: Bjoern Kaufmann To: isar-users Message-Id: In-Reply-To: References: <0a3ee875-d004-42d4-906b-4e6e847c7cd7@ilbers.de> <86320af4-266d-4040-9f41-6d2d9f267015n@googlegroups.com> <5850e65d-9636-47a9-847d-b6d2462353f9n@googlegroups.com> <2cb96a28-8df6-47c2-b16f-a8379d4ae6dc@ilbers.de> Subject: Re: No network available during task do_install on debian bullseye/5.10 host - but on a debian buster/4.19 host network is available MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_462128_366985227.1710770303608" X-TUID: +sIb+QKHbOys ------=_Part_462128_366985227.1710770303608 Content-Type: multipart/alternative; boundary="----=_Part_462129_1188754714.1710770303608" ------=_Part_462129_1188754714.1710770303608 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thanks for your clarification, that explains it. =20 Meanwhile I also found=20 https://github.com/ilbers/isar/blob/master/bitbake/lib/bb/utils.py#L1630=20 which is most probably the function responsible for disabling the network= =20 for tasks. But I was still wondering because the isar commit=20 (d26660b724b034b602f3889f55a23cd9be2e87bd) I though I was referencing in my= =20 build doesn't contain that function yet and also the whole [network]=20 functionality is missing. Turns out that I made a mistake when backtracking= =20 the commits of dependent layers of my build and I am actually using a=20 different isar commit (93cc388638336997a7c00b6ef8a58ee349407a54), which=20 already contains that functionality. =20 I tried it out again with do_testtask[network] =3D "1" and now the network= =20 interfaces are indeed available. Thank you all for your help. =20 Best regards, Bjoern On Friday, March 15, 2024 at 10:28:34=E2=80=AFAM UTC+1 Schmidt, Adriaan wro= te: > Anton Mikanovich, Sent: Friday, March 15, 2024 10:17 AM: > > 15/03/2024 11:06, Bjoern Kaufmann wrote: > > > I did what you proposed, but there is still no eth0. > > > What I also tested and what might be interesting: > > > > > > def print_ifs(): > > > import subprocess > > > import socket > > > > > > output =3D subprocess.check_output("ip a", shell=3DTrue) > > > print(f'Output of ip a: "{str(output)}"') > > > > > > print(socket.if_nameindex()) > > > return '' > > > > > > do_testtask() { > > > ${@ print_ifs()} > > > ip a > > > } > > > addtask testtask > > > > > > > > > I executed it inside kas shell by 'bitbake -c testtask my-recipe' > > > again and the log looks as follows: > > > > > > DEBUG: Executing shell function do_testtask > > > Output of ip a: "b'1: lo: mtu 65536 qdisc > > > noqueue state UNKNOWN group default qlen 1000\n link/loopback > > > 00:00:00:00:00:00 brd 00:00:00:00:00:00\n inet 127.0.0.1/8 scope > > > host lo\n valid_lft forever preferred_lft forever\n4: eth0@if5: > > > mtu 1500 qdisc noqueue state UP > > > group default \n link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:f= f > > > link-netnsid 0\n inet 172.17.0.2/16 brd 172.17.255.255 scope globa= l > > > eth0\n valid_lft forever preferred_lft forever\n'" > > > [(1, 'lo'), (4, 'eth0')] > > > Output of ip a: "b'1: lo: mtu 65536 qdisc noop state DOWN > > > group default qlen 1000\n link/loopback 00:00:00:00:00:00 brd > > > 00:00:00:00:00:00\n'" > > > [(1, 'lo')] > > > 1: lo: mtu 65536 qdisc noop state DOWN group default qlen= =20 > 1000 > > > link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 > > > DEBUG: Shell function do_testtask finished > > > > > > > > > So as you can see > > > 1. The python function is printed twice in a row, most probably in tw= o > > > different contexts? I guess you know more about it > > > 2. During the first execution of the python function, eth0 interfaces > > > are available > > > 3. During the second execution of the python function, no eth0 > > > interface is available > > > > > > > > > Also Jan Kiszka told me that to his knowledge the newer bitbake > > > isolates tasks from networks by default. If this is the case it still > > > doesn't really explain the behavior show in the log above and it > > > doesn't explain why this doesn't happen on the buster host VMs. > > > > > > Best regards, > > > Bjoern > >=20 > > Hello Bjoern, > >=20 > > The first print_ifs execution was done during recipe parsing, the secon= d=20 > one > > was done during task execution. > > It happens because you've used inline python call. > >=20 > > For bitbake 2.0+ you can enable network access for your task by setting= : > > do_testtask[network] =3D "1" > > Just to expand on this: In general, there is no networking in Bitbake=20 > tasks. > > From the Bitbake manual ( > https://docs.yoctoproject.org/bitbake/2.6/bitbake-user-manual/bitbake-use= r-manual-metadata.html#variable-flags > ): > =3D=3D=3D > Variable Flags > [...] > [network]: When set to =E2=80=9C1=E2=80=9D, allows a task to access the n= etwork. By=20 > default, only the do_fetch task is granted network access. Recipes=20 > shouldn=E2=80=99t access the network outside of do_fetch as it usually un= dermines=20 > fetcher source mirroring, image and licence manifests, software auditing= =20 > and supply chain security. > =3D=3D=3D > > Yocto changelog (https://docs.yoctoproject.org/singleindex.html, grep for= =20 > "[network]"): > =3D=3D=3D > Network access from tasks is now disabled by default on kernels which=20 > support this feature (on most recent distros such as CentOS 8 and Debian = 11=20 > onwards). This means that tasks accessing the network need to be marked a= s=20 > such with the network flag. For example: > > do_mytask[network] =3D "1" > This is allowed by default from do_fetch but not from any of our other=20 > standard tasks. Recipes shouldn=E2=80=99t be accessing the network outsid= e of=20 > do_fetch as it usually undermines fetcher source mirroring, image and=20 > licence manifests, software auditing and supply chain security. > =3D=3D=3D > > Note that the changelog mentions "Debian 11 onwards", which is why you ma= y=20 > be seeing a different behavior on buster. > > In addition for Isar: > The way the Bitbake feature is implemented has a side-effect that also=20 > disables sudo. So in Isar, "network" is also enabled for tasks that need= =20 > sudo. > > Adriaan > > > > On my side even without it 'ip a' was showing eth0, but there maybe som= e > > other > > permissions configuration. > >=20 > > -- > > You received this message because you are subscribed to the Google Grou= ps > > "isar-users" group. > > To unsubscribe from this group and stop receiving emails from it, send = an > > email to isar-users+...@googlegroups.com. > > To view this discussion on the web visit > > https://groups.google.com/d/msgid/isar-users/2cb96a28-8df6-47c2-b16f- > > a8379d4ae6dc%40ilbers.de. > ------=_Part_462129_1188754714.1710770303608 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

Thanks for= your clarification, that explains it.

=C2=A0

Meanwhile = I also found=C2=A0https://github.com/ilbers/isar/blob/master/bitbake/lib= /bb/utils.py#L1630 which is most probably the function responsible for disabling the network f= or tasks. But I was still wondering because the isar commit (d26660b724b034b602f3889f55a23cd9be2e87bd) I though I was referencing in my build doesn't contain that function yet and also the whole [network] functionality is missing. Turns out that I made a mistake when backtracking= the commits of dependent layers of my build and I am actually using a different isar commit (93cc388638336997a7c00b6ef8a58ee349407a54), which already conta= ins that functionality.

=C2=A0

I tried it= out again with=C2=A0do_testtask[network] =3D "1" and now the network interfaces are indeed available.

Thank you all for your help.


=C2=A0

Best regar= ds,

Bjoern

=
On = Friday, March 15, 2024 at 10:28:34=E2=80=AFAM UTC+1 Schmidt, Adriaan wrote:=
Anton Mikano= vich, Sent: Friday, March 15, 2024 10:17 AM:
> 15/03/2024 11:06, Bjoern Kaufmann wrote:
> > I did what you proposed, but there is still no eth0.
> > What I also tested and what might be interesting:
> >
> > def print_ifs():
> > =C2=A0 =C2=A0 import subprocess
> > =C2=A0 =C2=A0 import socket
> >
> > =C2=A0 =C2=A0 output =3D subprocess.check_output("ip a&q= uot;, shell=3DTrue)
> > =C2=A0 =C2=A0 print(f'Output of ip a: "{str(output)}= "')
> >
> > =C2=A0 =C2=A0 print(socket.if_nameindex())
> > =C2=A0 =C2=A0 return ''
> >
> > do_testtask() {
> > =C2=A0 =C2=A0 ${@ print_ifs()}
> > =C2=A0 =C2=A0 ip a
> > }
> > addtask testtask
> >
> >
> > I executed it inside kas shell by 'bitbake -c testtask my= -recipe'
> > again and the log looks as follows:
> >
> > DEBUG: Executing shell function do_testtask
> > Output of ip a: "b'1: lo: <LOOPBACK,UP,LOWER_UP&g= t; mtu 65536 qdisc
> > noqueue state UNKNOWN group default qlen 1000\n =C2=A0link/lo= opback
> > 00:00:00:00:00:00 brd 00:00:00:00:00:00\n =C2=A0 =C2=A0inet <= a href=3D"http://127.0.0.1/8" target=3D"_blank" rel=3D"nofollow" data-safer= edirecturl=3D"https://www.google.com/url?hl=3Den&q=3Dhttp://127.0.0.1/8= &source=3Dgmail&ust=3D1710834852028000&usg=3DAOvVaw2z90lmWQ-O8W= KI00oWYfVo">127.0.0.1/8 scope
> > host lo\n =C2=A0 =C2=A0 =C2=A0 valid_lft forever preferred_lf= t forever\n4: eth0@if5:
> > <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueu= e state UP
> > group default \n =C2=A0 =C2=A0link/ether 02:42:ac:11:00:02 br= d ff:ff:ff:ff:ff:ff
> > link-netnsid 0\n =C2=A0 =C2=A0inet 172.17.0.= 2/16 brd 172.17.255.255 scope global
> > eth0\n =C2=A0 =C2=A0 =C2=A0 valid_lft forever preferred_lft f= orever\n'"
> > [(1, 'lo'), (4, 'eth0')]
> > Output of ip a: "b'1: lo: <LOOPBACK> mtu 65536= qdisc noop state DOWN
> > group default qlen 1000\n =C2=A0 =C2=A0link/loopback 00:00:00= :00:00:00 brd
> > 00:00:00:00:00:00\n'"
> > [(1, 'lo')]
> > 1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group= default qlen 1000
> > =C2=A0 =C2=A0 link/loopback 00:00:00:00:00:00 brd 00:00:00:00= :00:00
> > DEBUG: Shell function do_testtask finished
> >
> >
> > So as you can see
> > 1. The python function is printed twice in a row, most probab= ly in two
> > different contexts? I guess you know more about it
> > 2. During the first execution of the python function, eth0 in= terfaces
> > are available
> > 3. During the second execution of the python function, no eth= 0
> > interface is available
> >
> >
> > Also Jan Kiszka told me that to his knowledge the newer bitba= ke
> > isolates tasks from networks by default. If this is the case = it still
> > doesn't really explain the behavior show in the log above= and it
> > doesn't explain why this doesn't happen on the buster= host VMs.
> >
> > Best regards,
> > Bjoern
>=20
> Hello Bjoern,
>=20
> The first print_ifs execution was done during recipe parsing, the = second one
> was done during task execution.
> It happens because you've used inline python call.
>=20
> For bitbake 2.0+ you can enable network access for your task by se= tting:
> do_testtask[network] =3D "1"

Just to expand on this: In general, there is no networking in Bitbake t= asks.

From the Bitbake manual (https://docs.yoctoproject.org/bitbake/2.6/bitbake-user-manual/bitbake-us= er-manual-metadata.html#variable-flags):
=3D=3D=3D
Variable Flags
[...]
[network]: When set to =E2=80=9C1=E2=80=9D, allows a task to access the= network. By default, only the do_fetch task is granted network access. Rec= ipes shouldn=E2=80=99t access the network outside of do_fetch as it usually= undermines fetcher source mirroring, image and licence manifests, software= auditing and supply chain security.
=3D=3D=3D

Yocto changelog (https://docs.yoctoproject.org/singleindex.html, grep for= "[network]"):
=3D=3D=3D
Network access from tasks is now disabled by default on kernels which s= upport this feature (on most recent distros such as CentOS 8 and Debian 11 = onwards). This means that tasks accessing the network need to be marked as = such with the network flag. For example:

do_mytask[network] =3D "1"
This is allowed by default from do_fetch but not from any of our other = standard tasks. Recipes shouldn=E2=80=99t be accessing the network outside = of do_fetch as it usually undermines fetcher source mirroring, image and li= cence manifests, software auditing and supply chain security.
=3D=3D=3D

Note that the changelog mentions "Debian 11 onwards", which i= s why you may be seeing a different behavior on buster.

In addition for Isar:
The way the Bitbake feature is implemented has a side-effect that also = disables sudo. So in Isar, "network" is also enabled for tasks th= at need sudo.

Adriaan


> On my side even without it 'ip a' was showing eth0, but th= ere maybe some
> other
> permissions configuration.
>=20
> --
> You received this message because you are subscribed to the Google= Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, = send an
> email to isar-users+...= @googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/= d/msgid/isar-users/2cb96a28-8df6-47c2-b16f-
> a8379d4ae6dc%40ilbers.de.
------=_Part_462129_1188754714.1710770303608-- ------=_Part_462128_366985227.1710770303608--