Re: Isar build tree structure

[Claudius Heine <claudius.heine.ext@siemens.com>]

Hi,

On 09/05/2017 11:35 AM, Alexander Smirnov wrote:
> 
> 
> On 09/05/2017 12:28 PM, Claudius Heine wrote:
>> On 09/05/2017 11:13 AM, Alexander Smirnov wrote:
>>>
>>>
>>> On 09/05/2017 12:06 PM, Claudius Heine wrote:
>>>> Hi,
>>>>
>>>> On 09/05/2017 10:21 AM, Alexander Smirnov wrote:
>>>>> Hi all,
>>>>>
>>>>> I'd like to discuss build folders tree to somehow chose more 
>>>>> suitable approach.
>>>>>
>>>>> 1. How it's done at the moment.
>>>>>
>>>>> The folder tree contains the following main paths:
>>>>>
>>>>>   - tmp/work/${PP}
>>>>>
>>>>> It's a $WORKDIR for each recipe, here is stored bibtbake logs and 
>>>>> commands, and data for packages that doesn't require buildchroot 
>>>>> (for example, buildchroot and isar-image-base).
>>>>>
>>>>>   - tmp/work/buildchroot/${DISTRO}-${ARCH}/rootfs/home/build/${PN}
>>>>>
>>>>> This is the $BUILDROOT for packages, that require compilation in 
>>>>> chroot. It is used as destination folder for unpack task.
>>>>>
>>>>> Benefits:
>>>>>
>>>>>   - Maximal common OE-like folder structure for all kind of 
>>>>> recipes. You can find the information about specific package in 
>>>>> 'tmp/work/${PP}'. And it doesn't matter if this package uses dpkg 
>>>>> class, or not.
>>>>>
>>>>>   - After buildchroot cleanup, the build information is not loosed.
>>>>>
>>>>>   - For multiple buildchroots in project, there will be single 
>>>>> place for package meta information.
>>>>>
>>>>> 2. Remove $BUILDROOT.
>>>>>
>>>>> This would mean that $WORKDIR location may vary depending on recipe 
>>>>> type. So the recipes data will be distributed across buildchroots 
>>>>> and tmp/work dir.
>>>>>
>>>>> Benefits:
>>>>>
>>>>>   - Each recipe will use single folder for everything.
>>>>>
>>>>>
>>>>>
>>>>> So... Ideas and opinions? :-)
>>>>>
>>>>
>>>> Why not both benefits? Using bind mount to put the workdir of 
>>>> recipes into the buildchroot?
>>>
>>> Mount is very difficult to manage. For example if something failed 
>>> during build, it's hard to check in next build what is already 
>>> mounted and what is not yet.
>>>
>>> But your idea is good. Could we use symlinks instead?
>>
>> If you are in a chroot environment the paths of the symlinks are wrong 
>> since they reference a different root directory.
> 
> Yeah, you are right... So probably mount could be an option:
> 
> = do_unpack =
> 1. unpack to WORKDIR (tmp/work/${PP})
> 
> = do_build =
> 1. mount respective folder in buildchroot
> 2. build
> 3. unmount respective folder in buildchroot
> 
> This approach will be quite good compatible with OE.

If we go with proot then all this mounting could be done without 
requiring root or this build step. I could be done like this:

     $ proot -S ${BUILDROOT} -b ${WORKDIR}:${CHROOT_BUILDDIR} build.sh

One problem with proot is this:

     $ mkdir test
     $ cd test
     $ proot -0
     # id
     uid=0(root) gid=0(root) groups=0(root)
     # touch testfile
     # ls -al
     total 8
     drwxr-xr-x  2 root root 4096 Sep  5 13:02 .
     drwxrwxrwt 18 root root 4096 Sep  5 13:02 ..
     -rw-r--r--  1 root root    0 Sep  5 13:02 testfile
     # chown 123:123 testfile
     # ls -al
     total 8
     drwxr-xr-x  2 root root 4096 Sep  5 13:02 .
     drwxrwxrwt 18 root root 4096 Sep  5 13:02 ..
     -rw-r--r--  1 root root    0 Sep  5 13:02 testfile
     # exit
     $ ls -al
     total 8
     drwxr-xr-x  2 user user 4096 Sep  5 13:02 .
     drwxrwxrwt 18 root root 4096 Sep  5 13:02 ..
     -rw-r--r--  1 user user    0 Sep  5 13:02 testfile

It maps just everything from the current user to root and back. So 
multiple users aren't supported here. But maybe this can be patched.

>>>>
>>>> Maybe we should take a look at proot [1] and use this instead of 
>>>> chroot, because its more flexible, does not require privileges, etc.
>>>>
>>>> [1] https://github.com/proot-me/PRoot/blob/master/doc/proot/manual.txt
>>>>
>>>
>>> Do you know if it fully supports dpkg & Co?
>>
>> I currently use proot in some parts of my isar fork already. And from 
>> my experience it works rather well. Since it uses the ptrace 
>> systemcall, static binaries work work as well.
>>
> 
> Do you have patch for this, could you please send it as RFC?

I don't have anything worth submitting (even as RFC) concerning proot. 
But AFAIK Johannes has converted more of isar to use proot. Maybe he can 
give some input for this.

Claudius


-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de